DCGAN Tutorial
==============

**Author**: `Nathan Inkawhich <https://github.com/inkawhich>`__

Introduction
------------

This tutorial will give an introduction to DCGANs through an example. We
will train a generative adversarial network (GAN) to generate new
celebrities after showing it pictures of many real celebrities. Most of
the code here is from the dcgan implementation in
`pytorch/examples <https://github.com/pytorch/examples>`__, and this
document will give a thorough explanation of the implementation and shed
light on how and why this model works. But don’t worry, no prior
knowledge of GANs is required, but it may require a first-timer to spend
some time reasoning about what is actually happening under the hood.
Also, for the sake of time it will help to have a GPU, or two. Lets
start from the beginning.

Generative Adversarial Networks
-------------------------------

What is a GAN?
~~~~~~~~~~~~~~

GANs are a framework for teaching a DL model to capture the training
data’s distribution so we can generate new data from that same
distribution. GANs were invented by Ian Goodfellow in 2014 and first
described in the paper `Generative Adversarial
Nets <https://papers.nips.cc/paper/5423-generative-adversarial-nets.pdf>`__.
They are made of two distinct models, a *generator* and a
*discriminator*. The job of the generator is to spawn ‘fake’ images that
look like the training images. The job of the discriminator is to look
at an image and output whether or not it is a real training image or a
fake image from the generator. During training, the generator is
constantly trying to outsmart the discriminator by generating better and
better fakes, while the discriminator is working to become a better
detective and correctly classify the real and fake images. The
equilibrium of this game is when the generator is generating perfect
fakes that look as if they came directly from the training data, and the
discriminator is left to always guess at 50% confidence that the
generator output is real or fake.

Now, lets define some notation to be used throughout tutorial starting
with the discriminator. Let $x$ be data representing an image.
$D(x)$ is the discriminator network which outputs the (scalar)
probability that $x$ came from training data rather than the
generator. Here, since we are dealing with images, the input to
$D(x)$ is an image of CHW size 3x64x64. Intuitively, $D(x)$
should be HIGH when $x$ comes from training data and LOW when
$x$ comes from the generator. $D(x)$ can also be thought of
as a traditional binary classifier.

For the generator’s notation, let $z$ be a latent space vector
sampled from a standard normal distribution. $G(z)$ represents the
generator function which maps the latent vector $z$ to data-space.
The goal of $G$ is to estimate the distribution that the training
data comes from ($p_{data}$) so it can generate fake samples from
that estimated distribution ($p_g$).

So, $D(G(z))$ is the probability (scalar) that the output of the
generator $G$ is a real image. As described in `Goodfellow’s
paper <https://papers.nips.cc/paper/5423-generative-adversarial-nets.pdf>`__,
$D$ and $G$ play a minimax game in which $D$ tries to
maximize the probability it correctly classifies reals and fakes
($logD(x)$), and $G$ tries to minimize the probability that
$D$ will predict its outputs are fake ($log(1-D(G(z)))$).
From the paper, the GAN loss function is

\begin{align}\underset{G}{\text{min}} \underset{D}{\text{max}}V(D,G) = \mathbb{E}_{x\sim p_{data}(x)}\big[logD(x)\big] + \mathbb{E}_{z\sim p_{z}(z)}\big[log(1-D(G(z)))\big]\end{align}

In theory, the solution to this minimax game is where
$p_g = p_{data}$, and the discriminator guesses randomly if the
inputs are real or fake. However, the convergence theory of GANs is
still being actively researched and in reality models do not always
train to this point.

What is a DCGAN?
~~~~~~~~~~~~~~~~

A DCGAN is a direct extension of the GAN described above, except that it
explicitly uses convolutional and convolutional-transpose layers in the
discriminator and generator, respectively. It was first described by
Radford et. al. in the paper `Unsupervised Representation Learning With
Deep Convolutional Generative Adversarial
Networks <https://arxiv.org/pdf/1511.06434.pdf>`__. The discriminator
is made up of strided
`convolution <https://pytorch.org/docs/stable/nn.html#torch.nn.Conv2d>`__
layers, `batch
norm <https://pytorch.org/docs/stable/nn.html#torch.nn.BatchNorm2d>`__
layers, and
`LeakyReLU <https://pytorch.org/docs/stable/nn.html#torch.nn.LeakyReLU>`__
activations. The input is a 3x64x64 input image and the output is a
scalar probability that the input is from the real data distribution.
The generator is comprised of
`convolutional-transpose <https://pytorch.org/docs/stable/nn.html#torch.nn.ConvTranspose2d>`__
layers, batch norm layers, and
`ReLU <https://pytorch.org/docs/stable/nn.html#relu>`__ activations. The
input is a latent vector, $z$, that is drawn from a standard
normal distribution and the output is a 3x64x64 RGB image. The strided
conv-transpose layers allow the latent vector to be transformed into a
volume with the same shape as an image. In the paper, the authors also
give some tips about how to setup the optimizers, how to calculate the
loss functions, and how to initialize the model weights, all of which
will be explained in the coming sections.

```python
from __future__ import print_function
#%matplotlib inline
import argparse
import os
import random
import torch
import torch.nn as nn
import torch.nn.parallel
import torch.backends.cudnn as cudnn
import torch.optim as optim
import torch.utils.data
import torchvision.datasets as dset
import torchvision.transforms as transforms
import torchvision.utils as vutils
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.animation as animation
from IPython.display import HTML

# Set random seed for reproducibility
manualSeed = 999
#manualSeed = random.randint(1, 10000) # use if you want new results
print("Random Seed: ", manualSeed)
random.seed(manualSeed)
torch.manual_seed(manualSeed)
```

    Random Seed:  999

    <torch._C.Generator at 0x1b45c5aa030>

Inputs
------

Let’s define some inputs for the run:

-  **dataroot** - the path to the root of the dataset folder. We will
   talk more about the dataset in the next section
-  **workers** - the number of worker threads for loading the data with
   the DataLoader
-  **batch_size** - the batch size used in training. The DCGAN paper
   uses a batch size of 128
-  **image_size** - the spatial size of the images used for training.
   This implementation defaults to 64x64. If another size is desired,
   the structures of D and G must be changed. See
   `here <https://github.com/pytorch/examples/issues/70>`__ for more
   details
-  **nc** - number of color channels in the input images. For color
   images this is 3
-  **nz** - length of latent vector
-  **ngf** - relates to the depth of feature maps carried through the
   generator
-  **ndf** - sets the depth of feature maps propagated through the
   discriminator
-  **num_epochs** - number of training epochs to run. Training for
   longer will probably lead to better results but will also take much
   longer
-  **lr** - learning rate for training. As described in the DCGAN paper,
   this number should be 0.0002
-  **beta1** - beta1 hyperparameter for Adam optimizers. As described in
   paper, this number should be 0.5
-  **ngpu** - number of GPUs available. If this is 0, code will run in
   CPU mode. If this number is greater than 0 it will run on that number
   of GPUs

```python
# Root directory for dataset
dataroot = "data/celeba"

# Number of workers for dataloader
workers = 2

# Batch size during training
batch_size = 128

# Spatial size of training images. All images will be resized to this
#   size using a transformer.
image_size = 64

# Number of channels in the training images. For color images this is 3
nc = 3

# Size of z latent vector (i.e. size of generator input)
nz = 100

# Size of feature maps in generator
ngf = 64

# Size of feature maps in discriminator
ndf = 64

# Number of training epochs
num_epochs = 5

# Learning rate for optimizers
lr = 0.0002

# Beta1 hyperparam for Adam optimizers
beta1 = 0.5

# Number of GPUs available. Use 0 for CPU mode.
ngpu = 1
```

Data
----

In this tutorial we will use the `Celeb-A Faces
dataset <http://mmlab.ie.cuhk.edu.hk/projects/CelebA.html>`__ which can
be downloaded at the linked site, or in `Google
Drive <https://drive.google.com/drive/folders/0B7EVK8r0v71pTUZsaXdaSnZBZzg>`__.
The dataset will download as a file named *img_align_celeba.zip*. Once
downloaded, create a directory named *celeba* and extract the zip file
into that directory. Then, set the *dataroot* input for this notebook to
the *celeba* directory you just created. The resulting directory
structure should be:

::

   /path/to/celeba
       -> img_align_celeba
           -> 188242.jpg
           -> 173822.jpg
           -> 284702.jpg
           -> 537394.jpg
              ...

This is an important step because we will be using the ImageFolder
dataset class, which requires there to be subdirectories in the
dataset’s root folder. Now, we can create the dataset, create the
dataloader, set the device to run on, and finally visualize some of the
training data.

```python
# We can use an image folder dataset the way we have it setup.
# Create the dataset
dataset = dset.ImageFolder(root=dataroot,
                           transform=transforms.Compose([
                               transforms.Resize(image_size),
                               transforms.CenterCrop(image_size),
                               transforms.ToTensor(),
                               transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5)),
                           ]))
# Create the dataloader
dataloader = torch.utils.data.DataLoader(dataset, batch_size=batch_size,
                                         shuffle=True, num_workers=workers)

# Decide which device we want to run on
device = torch.device("cuda:0" if (torch.cuda.is_available() and ngpu > 0) else "cpu")

# Plot some training images
real_batch = next(iter(dataloader))
plt.figure(figsize=(8,8))
plt.axis("off")
plt.title("Training Images")
plt.imshow(np.transpose(vutils.make_grid(real_batch[0].to(device)[:64], padding=2, normalize=True).cpu(),(1,2,0)))
```

Implementation
--------------

With our input parameters set and the dataset prepared, we can now get
into the implementation. We will start with the weight initialization
strategy, then talk about the generator, discriminator, loss functions,
and training loop in detail.

Weight Initialization
~~~~~~~~~~~~~~~~~~~~~

From the DCGAN paper, the authors specify that all model weights shall
be randomly initialized from a Normal distribution with mean=0,
stdev=0.02. The ``weights_init`` function takes an initialized model as
input and reinitializes all convolutional, convolutional-transpose, and
batch normalization layers to meet this criteria. This function is
applied to the models immediately after initialization.

```python
# custom weights initialization called on netG and netD
def weights_init(m):
    classname = m.__class__.__name__
    if classname.find('Conv') != -1:
        nn.init.normal_(m.weight.data, 0.0, 0.02)
    elif classname.find('BatchNorm') != -1:
        nn.init.normal_(m.weight.data, 1.0, 0.02)
        nn.init.constant_(m.bias.data, 0)
```

Generator
~~~~~~~~~

The generator, $G$, is designed to map the latent space vector
($z$) to data-space. Since our data are images, converting
$z$ to data-space means ultimately creating a RGB image with the
same size as the training images (i.e. 3x64x64). In practice, this is
accomplished through a series of strided two dimensional convolutional
transpose layers, each paired with a 2d batch norm layer and a relu
activation. The output of the generator is fed through a tanh function
to return it to the input data range of $[-1,1]$. It is worth
noting the existence of the batch norm functions after the
conv-transpose layers, as this is a critical contribution of the DCGAN
paper. These layers help with the flow of gradients during training. An
image of the generator from the DCGAN paper is shown below.

.. figure:: /_static/img/dcgan_generator.png
   :alt: dcgan_generator

Notice, the how the inputs we set in the input section (*nz*, *ngf*, and
*nc*) influence the generator architecture in code. *nz* is the length
of the z input vector, *ngf* relates to the size of the feature maps
that are propagated through the generator, and *nc* is the number of
channels in the output image (set to 3 for RGB images). Below is the
code for the generator.

```python
# Generator Code

class Generator(nn.Module):
    def __init__(self, ngpu):
        super(Generator, self).__init__()
        self.ngpu = ngpu
        self.main = nn.Sequential(
            # input is Z, going into a convolution
            nn.ConvTranspose2d( nz, ngf * 8, 4, 1, 0, bias=False),
            nn.BatchNorm2d(ngf * 8),
            nn.ReLU(True),
            # state size. (ngf*8) x 4 x 4
            nn.ConvTranspose2d(ngf * 8, ngf * 4, 4, 2, 1, bias=False),
            nn.BatchNorm2d(ngf * 4),
            nn.ReLU(True),
            # state size. (ngf*4) x 8 x 8
            nn.ConvTranspose2d( ngf * 4, ngf * 2, 4, 2, 1, bias=False),
            nn.BatchNorm2d(ngf * 2),
            nn.ReLU(True),
            # state size. (ngf*2) x 16 x 16
            nn.ConvTranspose2d( ngf * 2, ngf, 4, 2, 1, bias=False),
            nn.BatchNorm2d(ngf),
            nn.ReLU(True),
            # state size. (ngf) x 32 x 32
            nn.ConvTranspose2d( ngf, nc, 4, 2, 1, bias=False),
            nn.Tanh()
            # state size. (nc) x 64 x 64
        )

    def forward(self, input):
        return self.main(input)
```

```python
features = next(iter(dataloader))[0]
print(tuple(features.size()))

```

    (128, 3, 64, 64)

```python
from torchkeras import summary
# features = next(iter(dataloader))[0]

summary(Generator(ngpu), input_shape=(100, 4, 4))
```

    ----------------------------------------------------------------
            Layer (type)               Output Shape         Param #
    ================================================================
       ConvTranspose2d-1            [-1, 512, 7, 7]         819,200
           BatchNorm2d-2            [-1, 512, 7, 7]           1,024
                  ReLU-3            [-1, 512, 7, 7]               0
       ConvTranspose2d-4          [-1, 256, 14, 14]       2,097,152
           BatchNorm2d-5          [-1, 256, 14, 14]             512
                  ReLU-6          [-1, 256, 14, 14]               0
       ConvTranspose2d-7          [-1, 128, 28, 28]         524,288
           BatchNorm2d-8          [-1, 128, 28, 28]             256
                  ReLU-9          [-1, 128, 28, 28]               0
      ConvTranspose2d-10           [-1, 64, 56, 56]         131,072
          BatchNorm2d-11           [-1, 64, 56, 56]             128
                 ReLU-12           [-1, 64, 56, 56]               0
      ConvTranspose2d-13          [-1, 3, 112, 112]           3,072
                 Tanh-14          [-1, 3, 112, 112]               0
    ================================================================
    Total params: 3,576,704
    Trainable params: 3,576,704
    Non-trainable params: 0
    ----------------------------------------------------------------
    Input size (MB): 0.006104
    Forward/backward pass size (MB): 9.187500
    Params size (MB): 13.644043
    Estimated Total Size (MB): 22.837646
    ----------------------------------------------------------------

```python

```

Now, we can instantiate the generator and apply the ``weights_init``
function. Check out the printed model to see how the generator object is
structured.

```python
# Create the generator
netG = Generator(ngpu).to(device)

# Handle multi-gpu if desired
if (device.type == 'cuda') and (ngpu > 1):
    netG = nn.DataParallel(netG, list(range(ngpu)))

# Apply the weights_init function to randomly initialize all weights
#  to mean=0, stdev=0.02.
netG.apply(weights_init)

# Print the model
print(netG)
```

Discriminator
~~~~~~~~~~~~~

As mentioned, the discriminator, $D$, is a binary classification
network that takes an image as input and outputs a scalar probability
that the input image is real (as opposed to fake). Here, $D$ takes
a 3x64x64 input image, processes it through a series of Conv2d,
BatchNorm2d, and LeakyReLU layers, and outputs the final probability
through a Sigmoid activation function. This architecture can be extended
with more layers if necessary for the problem, but there is significance
to the use of the strided convolution, BatchNorm, and LeakyReLUs. The
DCGAN paper mentions it is a good practice to use strided convolution
rather than pooling to downsample because it lets the network learn its
own pooling function. Also batch norm and leaky relu functions promote
healthy gradient flow which is critical for the learning process of both
$G$ and $D$.

Discriminator Code

```python
class Discriminator(nn.Module):
    def __init__(self, ngpu):
        super(Discriminator, self).__init__()
        self.ngpu = ngpu
        self.main = nn.Sequential(
            # input is (nc) x 64 x 64
            nn.Conv2d(nc, ndf, 4, 2, 1, bias=False),
            nn.LeakyReLU(0.2, inplace=True),
            # state size. (ndf) x 32 x 32
            nn.Conv2d(ndf, ndf * 2, 4, 2, 1, bias=False),
            nn.BatchNorm2d(ndf * 2),
            nn.LeakyReLU(0.2, inplace=True),
            # state size. (ndf*2) x 16 x 16
            nn.Conv2d(ndf * 2, ndf * 4, 4, 2, 1, bias=False),
            nn.BatchNorm2d(ndf * 4),
            nn.LeakyReLU(0.2, inplace=True),
            # state size. (ndf*4) x 8 x 8
            nn.Conv2d(ndf * 4, ndf * 8, 4, 2, 1, bias=False),
            nn.BatchNorm2d(ndf * 8),
            nn.LeakyReLU(0.2, inplace=True),
            # state size. (ndf*8) x 4 x 4
            nn.Conv2d(ndf * 8, 1, 4, 1, 0, bias=False),
            nn.Sigmoid()
        )

    def forward(self, input):
        return self.main(input)
```

Now, as with the generator, we can create the discriminator, apply the
``weights_init`` function, and print the model’s structure.

```python
# Create the Discriminator
netD = Discriminator(ngpu).to(device)

# Handle multi-gpu if desired
if (device.type == 'cuda') and (ngpu > 1):
    netD = nn.DataParallel(netD, list(range(ngpu)))

# Apply the weights_init function to randomly initialize all weights
#  to mean=0, stdev=0.2.
netD.apply(weights_init)

# Print the model
print(netD)
```

Loss Functions and Optimizers
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

With $D$ and $G$ setup, we can specify how they learn
through the loss functions and optimizers. We will use the Binary Cross
Entropy loss
(`BCELoss <https://pytorch.org/docs/stable/nn.html#torch.nn.BCELoss>`__)
function which is defined in PyTorch as:

\begin{align}\ell(x, y) = L = \{l_1,\dots,l_N\}^\top, \quad l_n = - \left[ y_n \cdot \log x_n + (1 - y_n) \cdot \log (1 - x_n) \right]\end{align}

Notice how this function provides the calculation of both log components
in the objective function (i.e. $log(D(x))$ and
$log(1-D(G(z)))$). We can specify what part of the BCE equation to
use with the $y$ input. This is accomplished in the training loop
which is coming up soon, but it is important to understand how we can
choose which component we wish to calculate just by changing $y$
(i.e. GT labels).

Next, we define our real label as 1 and the fake label as 0. These
labels will be used when calculating the losses of $D$ and
$G$, and this is also the convention used in the original GAN
paper. Finally, we set up two separate optimizers, one for $D$ and
one for $G$. As specified in the DCGAN paper, both are Adam
optimizers with learning rate 0.0002 and Beta1 = 0.5. For keeping track
of the generator’s learning progression, we will generate a fixed batch
of latent vectors that are drawn from a Gaussian distribution
(i.e. fixed_noise) . In the training loop, we will periodically input
this fixed_noise into $G$, and over the iterations we will see
images form out of the noise.

```python
# Initialize BCELoss function
criterion = nn.BCELoss()

# Create batch of latent vectors that we will use to visualize
#  the progression of the generator
fixed_noise = torch.randn(64, nz, 1, 1, device=device)

# Establish convention for real and fake labels during training
real_label = 1.
fake_label = 0.

# Setup Adam optimizers for both G and D
optimizerD = optim.Adam(netD.parameters(), lr=lr, betas=(beta1, 0.999))
optimizerG = optim.Adam(netG.parameters(), lr=lr, betas=(beta1, 0.999))
```

Training
~~~~~~~~

Finally, now that we have all of the parts of the GAN framework defined,
we can train it. Be mindful that training GANs is somewhat of an art
form, as incorrect hyperparameter settings lead to mode collapse with
little explanation of what went wrong. Here, we will closely follow
Algorithm 1 from Goodfellow’s paper, while abiding by some of the best
practices shown in `ganhacks <https://github.com/soumith/ganhacks>`__.
Namely, we will “construct different mini-batches for real and fake”
images, and also adjust G’s objective function to maximize
$logD(G(z))$. Training is split up into two main parts. Part 1
updates the Discriminator and Part 2 updates the Generator.

**Part 1 - Train the Discriminator**

Recall, the goal of training the discriminator is to maximize the
probability of correctly classifying a given input as real or fake. In
terms of Goodfellow, we wish to “update the discriminator by ascending
its stochastic gradient”. Practically, we want to maximize
$log(D(x)) + log(1-D(G(z)))$. Due to the separate mini-batch
suggestion from ganhacks, we will calculate this in two steps. First, we
will construct a batch of real samples from the training set, forward
pass through $D$, calculate the loss ($log(D(x))$), then
calculate the gradients in a backward pass. Secondly, we will construct
a batch of fake samples with the current generator, forward pass this
batch through $D$, calculate the loss ($log(1-D(G(z)))$),
and *accumulate* the gradients with a backward pass. Now, with the
gradients accumulated from both the all-real and all-fake batches, we
call a step of the Discriminator’s optimizer.

**Part 2 - Train the Generator**

As stated in the original paper, we want to train the Generator by
minimizing $log(1-D(G(z)))$ in an effort to generate better fakes.
As mentioned, this was shown by Goodfellow to not provide sufficient
gradients, especially early in the learning process. As a fix, we
instead wish to maximize $log(D(G(z)))$. In the code we accomplish
this by: classifying the Generator output from Part 1 with the
Discriminator, computing G’s loss *using real labels as GT*, computing
G’s gradients in a backward pass, and finally updating G’s parameters
with an optimizer step. It may seem counter-intuitive to use the real
labels as GT labels for the loss function, but this allows us to use the
$log(x)$ part of the BCELoss (rather than the $log(1-x)$
part) which is exactly what we want.

Finally, we will do some statistic reporting and at the end of each
epoch we will push our fixed_noise batch through the generator to
visually track the progress of G’s training. The training statistics
reported are:

-  **Loss_D** - discriminator loss calculated as the sum of losses for
   the all real and all fake batches ($log(D(x)) + log(1 - D(G(z)))$).
-  **Loss_G** - generator loss calculated as $log(D(G(z)))$
-  **D(x)** - the average output (across the batch) of the discriminator
   for the all real batch. This should start close to 1 then
   theoretically converge to 0.5 when G gets better. Think about why
   this is.
-  **D(G(z))** - average discriminator outputs for the all fake batch.
   The first number is before D is updated and the second number is
   after D is updated. These numbers should start near 0 and converge to
   0.5 as G gets better. Think about why this is.

**Note:** This step might take a while, depending on how many epochs you
run and if you removed some data from the dataset.

```python
# Training Loop

# Lists to keep track of progress
img_list = []
G_losses = []
D_losses = []
iters = 0

print("Starting Training Loop...")
# For each epoch
for epoch in range(num_epochs):
    # For each batch in the dataloader
    for i, data in enumerate(dataloader, 0):

        ############################
        # (1) Update D network: maximize log(D(x)) + log(1 - D(G(z)))
        ###########################
        ## Train with all-real batch
        netD.zero_grad()
        # Format batch
        real_cpu = data[0].to(device)
        b_size = real_cpu.size(0)
        label = torch.full((b_size,), real_label, dtype=torch.float, device=device)
        # Forward pass real batch through D
        output = netD(real_cpu).view(-1)
        # Calculate loss on all-real batch
        errD_real = criterion(output, label)
        # Calculate gradients for D in backward pass
        errD_real.backward()
        D_x = output.mean().item()

        ## Train with all-fake batch
        # Generate batch of latent vectors
        noise = torch.randn(b_size, nz, 1, 1, device=device)
        # Generate fake image batch with G
        fake = netG(noise)
        label.fill_(fake_label)
        # Classify all fake batch with D
        output = netD(fake.detach()).view(-1)
        # Calculate D's loss on the all-fake batch
        errD_fake = criterion(output, label)
        # Calculate the gradients for this batch, accumulated (summed) with previous gradients
        errD_fake.backward()
        D_G_z1 = output.mean().item()
        # Compute error of D as sum over the fake and the real batches
        errD = errD_real + errD_fake
        # Update D
        optimizerD.step()

        ############################
        # (2) Update G network: maximize log(D(G(z)))
        ###########################
        netG.zero_grad()
        label.fill_(real_label)  # fake labels are real for generator cost
        # Since we just updated D, perform another forward pass of all-fake batch through D
        output = netD(fake).view(-1)
        # Calculate G's loss based on this output
        errG = criterion(output, label)
        # Calculate gradients for G
        errG.backward()
        D_G_z2 = output.mean().item()
        # Update G
        optimizerG.step()

        # Output training stats
        if i % 50 == 0:
            print('[%d/%d][%d/%d]\tLoss_D: %.4f\tLoss_G: %.4f\tD(x): %.4f\tD(G(z)): %.4f / %.4f'
                  % (epoch, num_epochs, i, len(dataloader),
                     errD.item(), errG.item(), D_x, D_G_z1, D_G_z2))

        # Save Losses for plotting later
        G_losses.append(errG.item())
        D_losses.append(errD.item())

        # Check how the generator is doing by saving G's output on fixed_noise
        if (iters % 500 == 0) or ((epoch == num_epochs-1) and (i == len(dataloader)-1)):
            with torch.no_grad():
                fake = netG(fixed_noise).detach().cpu()
            img_list.append(vutils.make_grid(fake, padding=2, normalize=True))

        iters += 1
```

Results
-------

Finally, lets check out how we did. Here, we will look at three
different results. First, we will see how D and G’s losses changed
during training. Second, we will visualize G’s output on the fixed_noise
batch for every epoch. And third, we will look at a batch of real data
next to a batch of fake data from G.

**Loss versus training iteration**

Below is a plot of D & G’s losses versus training iterations.

```python
plt.figure(figsize=(10,5))
plt.title("Generator and Discriminator Loss During Training")
plt.plot(G_losses,label="G")
plt.plot(D_losses,label="D")
plt.xlabel("iterations")
plt.ylabel("Loss")
plt.legend()
plt.show()
```

**Visualization of G’s progression**

Remember how we saved the generator’s output on the fixed_noise batch
after every epoch of training. Now, we can visualize the training
progression of G with an animation. Press the play button to start the
animation.

```python
#%%capture
fig = plt.figure(figsize=(8,8))
plt.axis("off")
ims = [[plt.imshow(np.transpose(i,(1,2,0)), animated=True)] for i in img_list]
ani = animation.ArtistAnimation(fig, ims, interval=1000, repeat_delay=1000, blit=True)

HTML(ani.to_jshtml())
```

**Real Images vs. Fake Images**

Finally, lets take a look at some real images and fake images side by
side.

```python
# Grab a batch of real images from the dataloader
real_batch = next(iter(dataloader))

# Plot the real images
plt.figure(figsize=(15,15))
plt.subplot(1,2,1)
plt.axis("off")
plt.title("Real Images")
plt.imshow(np.transpose(vutils.make_grid(real_batch[0].to(device)[:64], padding=5, normalize=True).cpu(),(1,2,0)))

# Plot the fake images from the last epoch
plt.subplot(1,2,2)
plt.axis("off")
plt.title("Fake Images")
plt.imshow(np.transpose(img_list[-1],(1,2,0)))
plt.show()
```

Where to Go Next
----------------

We have reached the end of our journey, but there are several places you
could go from here. You could:

-  Train for longer to see how good the results get
-  Modify this model to take a different dataset and possibly change the
   size of the images and the model architecture
-  Check out some other cool GAN projects
   `here <https://github.com/nashory/gans-awesome-applications>`__
-  Create GANs that generate
   `music <https://deepmind.com/blog/wavenet-generative-model-raw-audio/>`__


# Tutorial 2: Out-of-distribution (OOD) Learning

**Week 3, Day 4: Continual Learning**

**By Neuromatch Academy**

__Content creators:__  Avishree Khare, Het Shah, Joshua Vogelstein

__Content reviewers:__ Arush Tagade, Jeremy Forest, Kelson Shilling-Scrivo

__Content editors:__ Anoop Kulkarni, Spiros Chavlis

__Production editors:__ Deepak Raya, Spiros Chavlis

**Our 2021 Sponsors, including Presenting Sponsor Facebook Reality Labs**

---
# Tutorial Objectives

Deep Learning has seen tremendous growth in recent years thanks to more data, more compute and *very* deep neural networks. Although these networks perform extremely well on the specified task, they fail to generalize to newer tasks.

In this tutorial, we will explore Out-of-distribution (OOD) Learning and the several OOD paradigms that have been gaining popularity in recent years. We will understand what OOD really means and how it is different from anything else that we've looked at so far. We'll also take a look at Transfer Learning, Multi-task Learning and Meta-Learning which aim to facilitate OOD Learning in different ways. Here is a list of topics that we would be covering in this tutorial:

1. Introduction to OOD Learning
2. Transfer Learning
3. Multi-Task Learning
4. Meta-Learning

```python
# @title Tutorial slides

from IPython.display import IFrame
IFrame(src=f"https://mfr.ca-1.osf.io/render?url=https://osf.io/dqnm6/?direct%26mode=render%26action=download%26mode=render", width=854, height=480)
```

These are the slides for all videos in this tutorial. If you want to locally download the slides, click [here](https://osf.io/dqnm6/download).

---
# Setup

```python
# @title Install dependencies
!pip install Pillow --quiet
!pip install pandas --quiet
!pip3 install torch==1.9.0+cu111 torchvision==0.10.0+cu111 torchaudio==0.9.0 -f https://download.pytorch.org/whl/torch_stable.html  --quiet

!pip install git+https://github.com/NeuromatchAcademy/evaltools --quiet
from evaltools.airtable import AirtableForm

# generate airtable form
atform = AirtableForm('appn7VdPRseSoMXEG','W3D4_T2','https://portal.neuromatchacademy.org/api/redirect/to/1d7fcd5d-f1e9-4ac5-ae58-b0ade54a4f87')
```

```python
# Imports
import os
import copy
import time
import torch
import torchvision

import numpy as np
import matplotlib.pyplot as plt

import torch.nn as nn
import torch.optim as optim
import torch.nn.functional as F
from torch.optim import lr_scheduler
from torch.utils.data import Dataset, DataLoader, random_split

from torchvision.datasets import CelebA, Omniglot
from torchvision import datasets, models, transforms
```

```python
# @title Figure settings
import ipywidgets as widgets       # interactive display
%matplotlib inline
%config InlineBackend.figure_format = 'retina'
plt.style.use("https://raw.githubusercontent.com/NeuromatchAcademy/content-creation/main/nma.mplstyle")
```

```python
# @title Plotting functions

def visualize_siamese_sample(sample):
  to_PIL = transforms.ToPILImage()
  fig, axs = plt.subplots(1, 2, figsize=(8, 10))
  x1, x2, y = sample
  label = y.item()
  similarity = "Same character" if label == 1.0 else "Different characters"
  print(f"-------------------- Label: {label} ({similarity}) -------------------")
  axs[0].imshow(to_PIL(x1), cmap='gray')
  axs[1].imshow(to_PIL(x2), cmap='gray')
  # axs[0].set_title(f'Label: {label} ({similarity})')

  for i in range(2):
    axs[i].xaxis.set_ticks([])
    axs[i].yaxis.set_ticks([])

  plt.tight_layout()
  plt.show()

def visualize_mtfl_sample(sample):
  to_PIL = transforms.ToPILImage()
  img, labels = sample

  fig, ax = plt.subplots()
  ax.xaxis.set_ticks([])
  ax.yaxis.set_ticks([])
  print(f"Labels: {labels.tolist()}")
  ax.imshow(to_PIL(img))
  plt.show()

def visualize_one_shot_sample(sample):
  to_PIL = transforms.ToPILImage()
  query_img, support_imgs, support_labels, similarity = sample
  n_rows = 1
  n_cols = len(support_imgs) + 1
  fig, axs = plt.subplots(n_rows, n_cols, figsize=(n_cols*4, 10))

  axs[0].imshow(to_PIL(query_img), cmap='gray')
  axs[0].set_title('Query Image')
  axs[0].xaxis.set_ticks([])
  axs[0].yaxis.set_ticks([])

  for i, (s_img, s_label) in enumerate(zip(support_imgs, support_labels)):
    axs[i+1].imshow(to_PIL(s_img), cmap='gray')
    label = s_label.item()
    similarity = "Same character" if label == 1.0 else "Different characters"
    axs[i+1].set_title(f"Support Image {i+1} \n "
                       f"Label: {s_label.item()} ({similarity})")
    axs[i+1].xaxis.set_ticks([])
    axs[i+1].yaxis.set_ticks([])

  plt.tight_layout()
  plt.show()
```

```python
# @title Dataset Definitions and Helper Functions

# @markdown `pandas` and `PIL` libraries should be installed.
import pandas as pd
from PIL import Image

"""
Datasets for N-way 1-shot classification on the Omniglot Dataset
"""

class SiameseOmniglotDataset(Dataset):
  def __init__(self, num_samples=10000,
               data_transforms=transforms.ToTensor(), download=True):
    self.dataset = Omniglot(root='./data/',
                            background=True,
                            download=download,
                            transform=data_transforms)
    self.num_samples = num_samples
    self.num_classes = len(self.dataset._characters)
    self.classes = range(self.num_classes)
    self.instances_per_class = dict()
    self.get_instances_per_class()

  def __getitem__(self, idx):
    if idx % 2 == 0:
      return self.get_instances_from_same_class()
    return self.get_instances_from_diff_classes()

  def get_instances_from_same_class(self):
    c = random.randint(0, self.num_classes-1)
    [ch_1, ch_2] = random.sample(self.instances_per_class[c], 2)
    return ch_1, ch_2, torch.tensor([1.0])

  def get_instances_from_diff_classes(self):
    [c1, c2] = random.sample(self.classes, 2)
    [ch_1] = random.sample(self.instances_per_class[c1], 1)
    [ch_2] = random.sample(self.instances_per_class[c2], 1)
    return ch_1, ch_2, torch.tensor([0.0])

  def get_random_instance(self):
    ids = np.random.randint(0, len(self.dataset), 2)
    character_1 = self.dataset[ids[0]][0]
    character_2 = self.dataset[ids[1]][0]
    label_1 = self.dataset[ids[0]][1]
    label_2 = self.dataset[ids[1]][1]
    similarity = torch.tensor([label_1 == label_2], dtype=torch.float)
    return character_1, character_2, similarity

  def __len__(self):
    return self.num_samples

  def get_instances_per_class(self):
    for (image, label) in self.dataset:
      if label in self.instances_per_class:
        self.instances_per_class[label].append(image)
      else:
        self.instances_per_class[label] = [image]

class NWayOneShotOmniglotDataset(Dataset):
  def __init__(self, data_transforms=transforms.ToTensor(), n_ways=5,
               download=True):
    self.dataset = Omniglot(root='./data/',
                            background=False,
                            download=download,
                            transform=data_transforms)
    self.size = len(self.dataset)
    self.num_classes = len(self.dataset._characters)
    self.classes = range(self.num_classes)
    self.n_ways = n_ways
    self.instances_per_class = dict()
    self.get_instances_per_class()

  def __getitem__(self, idx):
    query_img, query_class = self.dataset[idx]

    # Find (n_ways - 1) distinct characters different from query_class
    support_imgs = []
    support_labels = []
    support_classes = random.sample([c for c in self.classes if c != query_class], self.n_ways-1)

    # Find 1 support image from query_class
    support_classes.append(query_class)

    random.shuffle(support_classes)

    for c in support_classes:
      [img] = random.sample(self.instances_per_class[c], 1)
      support_imgs.append(img)
      support_labels.append(torch.tensor([c == query_class], dtype=torch.float))

    _, similarity = torch.max(torch.tensor(support_labels), axis=0)
    return query_img, support_imgs, support_labels, similarity

  def __len__(self):
    return self.size

  def get_instances_per_class(self):
    for (image, label) in self.dataset:
      if label in self.instances_per_class:
        self.instances_per_class[label].append(image)
      else:
        self.instances_per_class[label] = [image]

class MTFLDataset(Dataset):
  def __init__(self, data_file,
               num_samples=10000,
               data_transforms=transforms.ToTensor(), ):

    self.df = pd.read_csv(data_file, sep=' ', header=None,
                          skipinitialspace=True, nrows=num_samples)
    self.df.iloc[:, 0] = self.df.iloc[:, 0].apply(lambda s: s.replace('\\', '/'))
    self.transform = data_transforms

  def __getitem__(self, idx):
    item = self.df.iloc[idx]
    img_name = item[0]
    labels = (item[11:] - 1)  # 1-indexed to 0-indexed

    img = Image.open(img_name)
    img = self.transform(img)

    return img, torch.from_numpy(np.array(labels, dtype=np.float32)).long()

  def __len__(self):
    return len(self.df)

"""
Load datasets
"""

def get_train_val_datasets(background_dataset_size=10000,
                           val_split=0.2, download=True):

  dataset_size = background_dataset_size
  val_split = 0.2
  train_size = int(dataset_size * (1 - val_split))
  val_size = dataset_size - train_size

  background_dataset = SiameseOmniglotDataset(num_samples=dataset_size,
                                              download=download)
  train_dataset, val_dataset = random_split(background_dataset, [train_size, val_size])

  return train_dataset, val_dataset

def get_test_dataset(n_ways=5, download=True):
  return NWayOneShotOmniglotDataset(n_ways=n_ways, download=download)

def get_train_val_datasets_mtfl(dirname, dataset_size=10000, val_split=0.2):

  data_transforms = transforms.Compose([transforms.Resize((256, 256)),
                                        transforms.ToTensor()])

  train_size = int(dataset_size * (1 - val_split))
  val_size = dataset_size - train_size

  background_dataset = MTFLDataset(data_file=f'{dirname}/MTFL/training.txt',
                                   num_samples=dataset_size,
                                   data_transforms=data_transforms)

  train_dataset, val_dataset = random_split(background_dataset, [train_size, val_size])

  return train_dataset, val_dataset
```

```python
# @title Training/Evaluation Helper Functions

# @markdown `train_multi_task(model, trainloader, valloader, criterion, optimizer, epochs=10, device='cpu')`
def train_multi_task(model, trainloader, valloader, criterion, optimizer,
                     epochs=10, device='cpu'):

  best_model_weights = copy.deepcopy(model.state_dict())
  best_overall_acc = 0.0

  for ep in range(epochs):
    print("")
    print("======== Epoch {:} / {:} ========".format(ep + 1, epochs))

    running_corrects = np.zeros(4)
    running_loss = 0.0
    model.train()
    model.to(device)
    for inps, labels in trainloader:
      inps = inps.to(device)
      labels = labels.to(device)
      outs = model(inps)
      overall_loss = 0.0

      for i in range(4):
        overall_loss += criterion(outs[i].to(device), labels[:, i].to(device))
        _, preds = torch.max(outs[i], 1)
        running_corrects[i] += torch.sum(preds == labels[:, i].data)

      running_loss += overall_loss.item()

      optimizer.zero_grad()
      overall_loss.backward()
      optimizer.step()

    train_loss = running_loss/(len(trainloader) * inps.size(0))
    train_accs = running_corrects / (len(trainloader) * inps.size(0))
    train_overall_acc = np.mean(train_accs)
    running_corrects_val = np.zeros(4)

    model.eval()
    with torch.no_grad():
      for inps, labels in valloader:
        inps = inps.to(device)
        labels = labels.to(device)
        outs = model(inps)

        overall_loss = 0.0

        for i in range(4):
          _, preds = torch.max(outs[i], 1)
          running_corrects_val[i] += torch.sum(preds == labels[:, i].data)

      val_accs = running_corrects_val / (len(valloader) * inps.size(0))

      val_overall_acc = np.mean(val_accs)

      if val_overall_acc > best_overall_acc:
        best_overall_acc = val_overall_acc
        best_model_weights = copy.deepcopy(model.state_dict())

    print(f"Training => Avg. Task Accuracy: {train_overall_acc}")
    print(f"Validation => Avg. Task Accuracy: {val_overall_acc}")

  model.load_state_dict(best_model_weights)
  return model

# @markdown `train_siamese_network(model, criterion, optimizer, train_loader, device='cpu', print_freq=100)`
def train_siamese_network(model, criterion, optimizer, train_loader,
                          device="cpu", print_freq=100):
  model.to(device)
  running_loss = 0.0
  correct = 0.0
  total = 0.0
  for batch_idx, data in enumerate(train_loader):
    x1 = data[0].to(device)
    x2 = data[1].to(device)
    y = data[2].to(device)

    optimizer.zero_grad() #Set parameter gradients to zero

    y_pred = model(x1, x2)
    # print(y_pred.shape, y.shape)
    loss = criterion(y_pred.view(-1), y.view(-1))
    loss.backward() #Set the values for gradients (`grad`) of all the parameters
    optimizer.step() #Update the parameter values using the computed gradients

    running_loss += loss.item()

    y_out = torch.round(torch.sigmoid(y_pred)) #To do: do necessary modifs from sigmoid to regular
    correct += y_out.eq(y.to(device)).sum()
    total += y_out.shape[0]

    # if (batch_idx % print_freq) == (print_freq - 1):
    #   print(f"Batch: {batch_idx} Running loss: {running_loss / print_freq} Train accuracy: {acc}")
    #   running_loss = 0.0

  print(f"Training => Average Loss: {running_loss/len(train_loader)} | Accuracy: {correct/total}")

# @markdown `evaluate_siamese_network(model, criterion, val_loader, device='cpu')`
def evaluate_siamese_network(model, criterion, val_loader, device='cpu'):
  model.eval()
  model.to(device)
  correct = 0.0
  total = 0.0
  val_loss = 0.0
  with torch.no_grad():
    for (x1, x2, y) in val_loader:
      y_pred = model(x1.to(device), x2.to(device))
      loss = criterion(y_pred.view(-1), y.to(device).view(-1))
      val_loss += loss.item()
      y_out = torch.round(torch.sigmoid(y_pred)) #To do: do necessary modifs from sigmoid to regular
      correct += y_out.eq(y.to(device)).sum()
      total += y_out.shape[0]

  val_acc = correct/total
  val_loss = val_loss/len(val_loader)

  print(f"Validation => Average Loss: {val_loss} | Accuracy: {val_acc}")

def evaluate_one_shot(model, test_loader, device='cpu'):
  model.eval()
  model.to(device)

  correct = 0.0
  total = 0.0

  with torch.no_grad():
    for (query_img, support_imgs, support_labels, similarity) in test_loader:
      x_query = query_img.to(device)
      y_pred_fc = [model(x_query, x_support.to(device)) for x_support in support_imgs]
      y_pred = [torch.sigmoid(pred) for pred in y_pred_fc]
      _, y_out = torch.max(torch.cat(y_pred, dim=1), 1)
      correct += y_out.eq(similarity.to(device)).sum()
      total += y_out.shape[0]

  print(f'Testing (One-shot) => Accuracy: {correct/total}')
```

```python
# @title Set random seed

# @markdown Executing `set_seed(seed=seed)` you are setting the seed

# for DL its critical to set the random seed so that students can have a
# baseline to compare their results to expected results.
# Read more here: https://pytorch.org/docs/stable/notes/randomness.html

# Call `set_seed` function in the exercises to ensure reproducibility.
import random
import torch

def set_seed(seed=None, seed_torch=True):
  if seed is None:
    seed = np.random.choice(2 ** 32)
  random.seed(seed)
  np.random.seed(seed)
  if seed_torch:
    torch.manual_seed(seed)
    torch.cuda.manual_seed_all(seed)
    torch.cuda.manual_seed(seed)
    torch.backends.cudnn.benchmark = False
    torch.backends.cudnn.deterministic = True

  print(f'Random seed {seed} has been set.')

# In case that `DataLoader` is used
def seed_worker(worker_id):
  worker_seed = torch.initial_seed() % 2**32
  np.random.seed(worker_seed)
  random.seed(worker_seed)
```

```python
# @title Set device (GPU or CPU). Execute `set_device()`
# especially if torch modules used.

# inform the user if the notebook uses GPU or CPU.

def set_device():
  device = "cuda" if torch.cuda.is_available() else "cpu"
  if device != "cuda":
    print("WARNING: For this notebook to perform best, "
        "if possible, in the menu under `Runtime` -> "
        "`Change runtime type.`  select `GPU` ")
  else:
    print("GPU is enabled in this notebook.")

  return device
```

```python
SEED = 2021
set_seed(seed=SEED)
DEVICE = set_device()
```

---
# Section 1: Introduction to Out-of-Distribution (OOD) Learning

*Time estimate: ~5mins*

In this section, we'll take a brief look at what OOD Learning is and how it can be useful in solving various problems with traditional Deep Learning.

```python
# @title Video 1: The Future of Learning
from ipywidgets import widgets

out2 = widgets.Output()
with out2:
  from IPython.display import IFrame
  class BiliVideo(IFrame):
    def __init__(self, id, page=1, width=400, height=300, **kwargs):
      self.id=id
      src = "https://player.bilibili.com/player.html?bvid={0}&page={1}".format(id, page)
      super(BiliVideo, self).__init__(src, width, height, **kwargs)

  video = BiliVideo(id=f"BV1764y167fj", width=854, height=480, fs=1)
  print("Video available at https://www.bilibili.com/video/{0}".format(video.id))
  display(video)

out1 = widgets.Output()
with out1:
  from IPython.display import YouTubeVideo
  video = YouTubeVideo(id=f"zMQySpHBoGs", width=854, height=480, fs=1, rel=0)
  print("Video available at https://youtube.com/watch?v=" + video.id)
  display(video)

out = widgets.Tab([out1, out2])
out.set_title(0, 'Youtube')
out.set_title(1, 'Bilibili')

# add event to airtable
atform.add_event('Video 1: The Future of Learning')

display(out)
```

## Think! 1.1: What are the problems that *you* want to solve?

Think of a problem that you would want to have solved using Deep Learning. Don't restrict yourself to the problems you have looked at so far!

Take 2 mins to think in silence and jot down a brief description of the problem here.

```python
# @markdown
Problem_that_you_want_to_solve = '' # @param {type:"string"}
```

Now discuss the following as a group:

1. Can these problems be solved using the techniques that you have looked at so far? If not, why?
2. Is there an OOD element attached to any of the problems, which makes them harder to solve?
3. Can you think of ways you could solve these problems? (Don't worry if you struggle, we'll hopefully find some suggestions by the end of this tutorial)

---
# Section 2: Transfer and Multi-Task Learning

*Time estimate: ~35mins*

In this tutorial we will learn transfer learning and multi-task learning. Transfer learning is a machine learning method where a model which is trained on a task (or dataset) can be used as a good initialization for training on a completely different and unrelated task (or dataset). Transfer learning is a very commonly used practice in a lot of Deep Learning works. This method is not limited to any particular subdomain and is widely used in almost all the domains, namely Computer Vision, Natural Language Processing, Reinforcement Learning, etc.

Multi-task learning aims to learn multiple tasks simultaneously using shared knowledge across these tasks. We can use Transfer learning to initialize the shared parts of the network.

We aim to learn these concepts via a simple problem of attributes classification for the celebA dataset.

```python
# @title Video 2: Transfer and Multi-Tasking Learning
from ipywidgets import widgets

out2 = widgets.Output()
with out2:
  from IPython.display import IFrame
  class BiliVideo(IFrame):
    def __init__(self, id, page=1, width=400, height=300, **kwargs):
      self.id=id
      src = "https://player.bilibili.com/player.html?bvid={0}&page={1}".format(id, page)
      super(BiliVideo, self).__init__(src, width, height, **kwargs)

  video = BiliVideo(id=f"BV18w41197o8", width=854, height=480, fs=1)
  print("Video available at https://www.bilibili.com/video/{0}".format(video.id))
  display(video)

out1 = widgets.Output()
with out1:
  from IPython.display import YouTubeVideo
  video = YouTubeVideo(id=f"PplDj0vXuIY", width=854, height=480, fs=1, rel=0)
  print("Video available at https://youtube.com/watch?v=" + video.id)
  display(video)

out = widgets.Tab([out1, out2])
out.set_title(0, 'Youtube')
out.set_title(1, 'Bilibili')

# add event to airtable
atform.add_event('Video 2: Transfer and Multi-Tasking Learning')

display(out)
```

## Section 2.1: Getting the data
We will be using the [MTFL](http://mmlab.ie.cuhk.edu.hk/projects/TCDCN.html) dataset for demonstrating the concepts of Transfer and Multi-Task Learning.

```python
# @title Get the current working directory in `CWD` variable.
CWD = os.getcwd()
print(f'Current dir: {CWD}')
```

```python
# @title Download and unzip the dataset
import requests, zipfile

# originally from 'http://mmlab.ie.cuhk.edu.hk/projects/TCDCN/data/MTFL.zip'
os.chdir(CWD)
name = 'MTFL'
fname = f"{name}.zip"
url = "https://osf.io/u5emj/download"

if not os.path.exists(name):
  print("Start downloading and unzipping `MTFL` dataset...")
  r = requests.get(url, allow_redirects=True)
  with open(fname, 'wb') as fd:
    fd.write(r.content)
  with zipfile.ZipFile(fname, 'r') as zip_ref:
    zip_ref.extractall(name)
  print("Download completed.")
else:
  print('Data has been already downloaded.')

print('Change working direcrtory!')
os.chdir(name)
print(f'Current dir: {os.getcwd()}')
```

Lets load the dataset into dataloaders which will help us in training the model!

You can check the implementation  of the `get_train_val_datasets_mtfl()` in the hidden cell `Dataset Definition and Helper Functions`

```python
"""
Create dataloaders for the train and validation datasets
"""

train_dataset, val_dataset = get_train_val_datasets_mtfl(CWD)

# Change this for a different batch size
batch_size = 16

g_seed = torch.Generator()
g_seed.manual_seed(SEED)

train_loader = DataLoader(
    dataset=train_dataset,
    batch_size=batch_size,
    num_workers=2,
    worker_init_fn=seed_worker,
    generator=g_seed
    )

val_loader = DataLoader(
    dataset=val_dataset,
    batch_size=batch_size,
    num_workers=2,
    worker_init_fn=seed_worker,
    generator=g_seed
    )
```

We visualize the samples from the dataset, we can see  that the input will be an image and there are multiple labels for the same image. The description of each label -

 - gender : 0 - male, 1 - female
 - smiling : 0 - yes, 1 - no
 - wearing glasses : 0 - yes, 1 - no
 - head pose : 0° - 0, +30° - 1, -30° - 2, +60° - 3, -60° - 4

```python
sample = train_dataset[0]
visualize_mtfl_sample(sample)
```

Can you interpret the above image and the labels corresponding to the image? Let's have a look at some more instances from the dataset.

## Section 2.2: Defining the model

A typical multi-task model looks like below, with shared layers being layers where knowledge is shared. Task-specific layers, as the name suggests, are specific to the tasks.

For the shared layers we will use a pre-trained backbone network. Here you can try out various models, we have provided an example of using resnet18 backbone. We have to remove the last fully connected layer, because that is specifically used for classificiation task. Futher, for each task we have some fully connected layers, you can play around with more layers on your own!

### Coding Exercise 2.2: Creating a Multi-Task model

Complete the custom model class `Multi_task_model`.
Complete the `forward()` function, by adding your solution to collect outputs for all the tasks using the fully connected for the corresponding task.

You can find an entire list of pre-trained models provided by PyTorch [here](https://pytorch.org/vision/stable/models.html)

```python
class Multi_task_model(nn.Module):
  def __init__(self, pretrained=True, num_tasks=4, load_file=None,
               num_labels_per_task=[2, 2, 2, 5]):
    super(Multi_task_model, self).__init__()
    self.backbone = models.resnet18(pretrained=pretrained)  # You can play around with different pre-trained models
    if load_file:
      self.backbone.load_state_dict(torch.load(load_file))
    self.backbone = torch.nn.Sequential(*(list(self.backbone.children())[:-1]))  # Remove the last fully connected layer

    if pretrained:
      for param in self.backbone.parameters():
        param.requires_grad = False

    self.fcs = []

    self.num_tasks = num_tasks

    for i in range(self.num_tasks):
      self.fcs.append(nn.Sequential(
          nn.Linear(512, 128),
          nn.ReLU(),
          nn.Dropout(0.4),
          nn.Linear(128, num_labels_per_task[i]),
          ################################
          # Add more layers if you want! #
          ################################
          nn.Softmax(dim=1),
      ))

      self.fcs = nn.ModuleList(self.fcs)

  def forward(self, x):
    x = self.backbone(x)
    x = torch.flatten(x, 1)
    outs = []
    #################################################
    # Collect outputs for each task
    # Fill in missing code below (...),
    # then remove or comment the line below to test your implementation
    raise NotImplementedError("Complete the model!")
    #################################################
    for i in range(...):
      outs.append(...)
    return outs

# add event to airtable
atform.add_event('Coding Exercise 2.2: Creating a Multi-Task model')
```

[*Click for solution*](https://github.com/NeuromatchAcademy/course-content-dl/tree/main//tutorials/W3D4_ContinualLearning/solutions/W3D4_Tutorial2_Solution_0a937c07.py)

To avoid any major blackout due to multiple downloads, we download a pretrained ResNet model locally.

**Note:** If `pretrained=False` and `load_file` exists, we load the pretrained model from a file. If you want to use a different model, set `load_file` to `None`.

```python
# @title Download `resnet18` pretrained
url = "https://osf.io/2kd98/download"
fname = "resnet18-f37072fd.pth"
r = requests.get(url, allow_redirects=True)

with open(fname, 'wb') as fd:
  fd.write(r.content)
```

```python
"""
Initialize two models with and without pre-trained weights
"""
model_with_pre_trained_backbone = Multi_task_model(pretrained=False,
                                                   load_file=fname).to(DEVICE)
model_without_pre_trained_backbone = Multi_task_model(pretrained=False).to(DEVICE)
```

Let's define the loss function and optimizer. Since this is a classification task, we will be using cross-entropy loss. For the optimizer we will be using Stochastic Gradient descent.

```python
criterion = nn.CrossEntropyLoss()

# Observe that all parameters are being optimized
optimizer_without_pre_trained_backbone = optim.SGD(
    model_without_pre_trained_backbone.parameters(),
    lr=0.001,
    momentum=0.9
    )

optimizer_pre_trained_backbone = optim.SGD(
    model_with_pre_trained_backbone.parameters(),
    lr=0.001,
    momentum=0.9
    )
```

## Section 2.3: Training model without pretrained backbone

```python
model_without_pre_trained_backbone = train_multi_task(
    model_without_pre_trained_backbone,
    train_loader,
    val_loader,
    criterion,
    optimizer_without_pre_trained_backbone,
    epochs=5, device=DEVICE
    )
```

## Section 2.4: Training model with pretrained backbone

```python
model_pre_trained_backbone = train_multi_task(model_with_pre_trained_backbone,
                                              train_loader,
                                              val_loader,
                                              criterion,
                                              optimizer_pre_trained_backbone,
                                              epochs=5, device=DEVICE)
```

## Section 2.5: Section Summary

We can summarize this section as follows

* Transfer learning helps in initializing the model better and thus makes learning of the model faster. It also gives better performance compared to the same model that was trained from scratch. Below is a table which contains validation accuracies after 5 epochs (your accuracies can be slightly different when you run the training loops. It will also depend on the number of epochs and the model you have implemented)-

|           Method          | Validation Accuracy |
|:-------------------------:|:-------------------:|
|   With Transfer Learning  |      **0.779**      |
| Without Transfer Learning |        0.728        |

* Multi-task learning helps us learn multiple tasks together, by sharing knowledge across layers. Task specific layers then help the model learn task specific features.

---
# Section 3: Interpretability

```python
# @title Video 3: Interpretability
from ipywidgets import widgets

out2 = widgets.Output()
with out2:
  from IPython.display import IFrame
  class BiliVideo(IFrame):
    def __init__(self, id, page=1, width=400, height=300, **kwargs):
      self.id=id
      src = "https://player.bilibili.com/player.html?bvid={0}&page={1}".format(id, page)
      super(BiliVideo, self).__init__(src, width, height, **kwargs)

  video = BiliVideo(id=f"BV1Ey4y1j7Jj", width=854, height=480, fs=1)
  print("Video available at https://www.bilibili.com/video/{0}".format(video.id))
  display(video)

out1 = widgets.Output()
with out1:
  from IPython.display import YouTubeVideo
  video = YouTubeVideo(id=f"P_QtnN1_wMM", width=854, height=480, fs=1, rel=0)
  print("Video available at https://youtube.com/watch?v=" + video.id)
  display(video)

out = widgets.Tab([out1, out2])
out.set_title(0, 'Youtube')
out.set_title(1, 'Bilibili')

# add event to airtable
atform.add_event('Video 3: Interpretability')

display(out)
```

Lets' find the most important feature for Cancer prediction and see if it belongs to the set of two most important features for the same problem!

## Section 3.1: Defining the problem setup

Imagine that you want to apply all the knowledge that you have gained from this course to a real health problem, say predicting cancer. You develop an algorithm for predicting cancer using certain input features from patient data.

This is a composite model that takes in three binary variables: does the patient have a family history of cancer (1=yes, 0=no), does the patient smoke (1=yes, 0=no), and is the patient young (age < 40) (1=yes, 0=no).

The algorithm does not favour either of the outcomes, so $P(cancer) = P(no\_cancer) = .5$

You wish to find the most important feature according to the model and you have the following list of conditional probabilities.

\begin{align}
P(family\_history = 1  | no\_cancer) = .1 &, P(family\_history = 1 | cancer) = .9 \\
P(smoker = 1 | no\_cancer) = .05 &, P(smoker = 1| cancer) = .8 \\
P(young = 1 | no\_cancer) = .01 &, P(young = 1 | cancer) = .71
\end{align}

## Section 3.2: Defining a feature importance metric

Let's define a metric that can help us quantify how important a feature (or a given set of features) is!

### Think! 3.2.1: Think of a suitable feature importance metric!

Take a moment to think about feature importance and a simple metric for quantifying it! Remember that the feature importance should depend on how useful the feature is for predicting the outcome *alone*.

```python
# @title Student Response
from ipywidgets import widgets

text=widgets.Textarea(
   value='Type your answer here and click on `Submit!`',
   placeholder='Type something',
   description='',
   disabled=False
)

button = widgets.Button(description="Submit!")

display(text,button)

def on_button_clicked(b):
   atform.add_answer('q1' , text.value)
   print("Submission successful!")

button.on_click(on_button_clicked)
```

Now discuss the following with the group:

1. Each member of the group should briefly talk about the feature metric they came up with and why they think it is suitable for the given problem setup.
2. Identify the most important characterics of a good feature importance metric. (As different members of the group used different criteria to select their metrics, find out the most important criteria that helped!). Try to limit yourself to the two most important characteristics.

Note down the most important characteristic here!

```python
# @title Student Response
from ipywidgets import widgets

text=widgets.Textarea(
   value='Type the most important characteristic and click on `Submit!`',
   placeholder='Type something',
   description='',
   disabled=False
)

button = widgets.Button(description="Submit!")

display(text,button)

def on_button_clicked(b):
   atform.add_answer('q2' , text.value)
   print("Submission successful!")

button.on_click(on_button_clicked)
```

Also note down the second most important characteristic here.

```python
# @title Student Response
from ipywidgets import widgets

text=widgets.Textarea(
   value='Type the second most important characteristic and click on `Submit!`',
   placeholder='Type something',
   description='',
   disabled=False
)

button = widgets.Button(description="Submit!")

display(text,button)

def on_button_clicked(b):
   atform.add_answer('q3' , text.value)
   print("Submission successful!")

button.on_click(on_button_clicked)
```

Thank you for coming up with some guidelines for what makes a feature importance metric good!

Although the metrics defined by you must be great, we here define a simple metric for maintaining consistency throughout the rest of this tutorial.

Let's assume that we wish to find the K best features. We know that a set of K features would be most important if they can predict the outcome with high confidence and better than any other set of the same size! We could hence try to predict the outcome using only the given set of features and record the accuracy. If this accuracy is higher than that using any other set of the same size, then we can say that the given set has the most important K features!

In essence, we would want to find the accuracies considering all feature sets of size K and then pick the one with the highest accuracy.

If we wish to define a loss instead (the lower the better), we could simply extend the above notion as follows:

\begin{equation}
Loss = 1 - Accuracy
\end{equation}

### Think! 3.2.2: Is our feature importance metric good?

Now that we have provided a new feature importance metric, think of the following:

1. What do you think of this new feature importance metric, and how is it different from the one that you came up with?
2. Does it satify the criteria that you came up with?
3. If not, why? Does the metric need to be modified or the criteria?

```python
# @title Student Response
from ipywidgets import widgets

text=widgets.Textarea(
   value='Type your answer here and click on `Submit!`',
   placeholder='Type something',
   description='',
   disabled=False
)

button = widgets.Button(description="Submit!")

display(text,button)

def on_button_clicked(b):
   atform.add_answer('q4' , text.value)
   print("Submission successful!")

button.on_click(on_button_clicked)
```

## Section 3.3: Finding the most important feature

Considering the feature importance metric that we defined above, let's try to find the most important and the two most important features!

The accuracy can be calculated using the probabilities specified above. Here are the accuracies for different feature sets! (Note that we have not detailed the calculation of these accuracies. If you are curious, refer to the bonus section!)

For a detailed explanation, see *Bonus 3*.

|           Feature(s)          | Accuracy |
|:---------------------------:|:-------------------:|
|   Family History            |        0.90         |
| Is a Smoker                 |        0.875        |
| Is Young                    |        0.85         |
| Family History, Is a Smoker |        0.917        |
| Is a Smoker, Is Young       |        0.941        |
| Family History, Is Young    |        0.931        |

### Think! 3.3.1: Finding important features

Given the table above, what is the single most important feature?

```python
# @title Student Response
from ipywidgets import widgets

text=widgets.Textarea(
   value='Type your answer here and click on `Submit!`',
   placeholder='Type something',
   description='',
   disabled=False
)

button = widgets.Button(description="Submit!")

display(text,button)

def on_button_clicked(b):
   atform.add_answer('q5' , text.value)
   print("Submission successful!")

button.on_click(on_button_clicked)
```

[*Click for solution*](https://github.com/NeuromatchAcademy/course-content-dl/tree/main//tutorials/W3D4_ContinualLearning/solutions/W3D4_Tutorial2_Solution_240f6cb7.py)

Can you also find out the second most important features?

```python
# @title Student Response
from ipywidgets import widgets

text=widgets.Textarea(
   value='Type your answer here and click on `Submit!`',
   placeholder='Type something',
   description='',
   disabled=False
)

button = widgets.Button(description="Submit!")

display(text,button)

def on_button_clicked(b):
   atform.add_answer('q6' , text.value)
   print("Submission successful!")

button.on_click(on_button_clicked)
```

[*Click for solution*](https://github.com/NeuromatchAcademy/course-content-dl/tree/main//tutorials/W3D4_ContinualLearning/solutions/W3D4_Tutorial2_Solution_2566d409.py)

The most important feature is not one of the two most important features! Can you find out why this is the case?

## Section 3.4: Section Summary

In this section, we discussed the following:

* What is Interpretability and why is it important?

* How can input features be used to understand the model's predictions (aka feature importances)?

---
# Summary

Recall the problem that you set out to solve at the beginning of this tutorial.

```python
# @markdown *Run me to print the problem that you wanted to solve!*
print(Problem_that_you_want_to_solve)
```

Take a moment to think about the following:

1. Did this tutorial help you find some ideas to solve this problem?
2. Can you now come up with a solution to this problem by using all the tools that you learnt from this course?

We sincerely hope that your answers to these two questions are yes and YES! Recall all that you learnt from this course and how this knowledge and practical experience can help with a problem that *you* want to solve. Go back and experiment with your solutions and see if you can solve this problem that matters to you!

**Congratulations! You have finished the NMA-DL course!**

```python
# @title Airtable Submission Link
from IPython import display as IPydisplay
IPydisplay.HTML(
   f"""
 <div>
   <a href= "{atform.url()}" target="_blank">
   </a>
   </div>""" )
```

---
# Bonus 1: Introduction to Meta Learning

*Time estimate: ~20mins*

In this section, we will *literally* learn to learn by exploring the concept of Meta Learning. Meta Learning attempts to improve the generalization capabilities of neural networks by teaching them *how to learn new tasks fast*. We aim to introduce you to the following topics:

1. Meta-Learning and its applications
2. Few-shot classification (the most common application of Meta-Learning in supervised classification settings)
3. One-shot Learning with Convolutional Siamese Networks

```python
# @title Video 4: Meta Learning
from ipywidgets import widgets

out2 = widgets.Output()
with out2:
  from IPython.display import IFrame
  class BiliVideo(IFrame):
    def __init__(self, id, page=1, width=400, height=300, **kwargs):
      self.id=id
      src = "https://player.bilibili.com/player.html?bvid={0}&page={1}".format(id, page)
      super(BiliVideo, self).__init__(src, width, height, **kwargs)

  video = BiliVideo(id=f"BV1zU4y1n77d", width=854, height=480, fs=1)
  print("Video available at https://www.bilibili.com/video/{0}".format(video.id))
  display(video)

out1 = widgets.Output()
with out1:
  from IPython.display import YouTubeVideo
  video = YouTubeVideo(id=f"8grBwtrzcDI", width=854, height=480, fs=1, rel=0)
  print("Video available at https://youtube.com/watch?v=" + video.id)
  display(video)

out = widgets.Tab([out1, out2])
out.set_title(0, 'Youtube')
out.set_title(1, 'Bilibili')

# add event to airtable
atform.add_event('Video 4: Meta Learning')

display(out)
```

Meta-Learning is most commonly observed as Few-shot Learning in Supervised learning settings.

Few-shot Learning aims to answer the following question: How can a neural network learn a task well with *very little* data?

In the context of supervised classification, K-shot learning refers to learning with only K examples of each class. An extension of this would be N-way K-shot learning, which attempts to train a network with only K examples of each of the N classes. Let's take an example of a 5-way 1-shot problem:

Now that we know what Few-shot learning is, let's take a look at a One-shot classification problem using Siamese Networks!

## Bonus Section 1.1: Introduction to Omniglot

The Omniglot data set is a a standardized benchmark for evaluating the performance of Few-shot Learning algorithms. It contains 1623 different handwritten characters from 50 different alphabets. The alphabets range from well-established international languages like Latin and Korean to lesser known local dialects. Fictitious character sets such as Aurek-Besh and Klingon are also included. Here are a few examples from the Omniglot dataset.

## Bonus Section 1.2: Convolutional Siamese Networks

The simplest way to evaluate where an image belongs to a class is by comparing it with other images of the same class. Convolutional Siamese networks help us in comparing two images and quantify how similar or different they are from one another. This is known as verification.

Verification of two images is done as follows: Both the images are passed through the convolutional network to generate *feature vectors* representing them. These feature vectors are then compared for similarity. (Find a better image for siamese network maybe)

How can we compare if two vectors are similar? By using a distance metric! **L1 distance** is one such metric that evaluates the distance between two vectors by computing the absolute value of the difference of invidual components of both vectors. More specifically, given vectors `v1` and `v2`, the component-wise L1 distance would be as follows:

```
l1_distance = abs(v1 - v2)
```

After this, the problem boils down to a binary classification problem where two images either belong to the same class or not!

### Bonus Coding Exercise 1.2: Creating a Convolutional Siamese Network

Let's create a Convolutional Siamese Network! The structure of the network is created in the `ConvSiameseNet` class. You have to compute the L1 distance (`l1_distance`) between the feature vectors (`x1_fv` and `x2_fv`). You can use `torch.abs()` to compute the absolute value of a tensor.

```python
# Define the Siamese Network
class ConvSiameseNet(nn.Module):
  """
  Convolutional Siamese Network from "Siamese Neural Networks for One-shot Image Recognition"
  Paper can be found at http://www.cs.toronto.edu/~rsalakhu/papers/oneshot1.pdf
  """

  def __init__(self):
    super().__init__()
    self.conv1 = nn.Conv2d(1, 64, 10)
    self.conv2 = nn.Conv2d(64, 128, 7)
    self.conv3 = nn.Conv2d(128, 128, 4)
    self.conv4 = nn.Conv2d(128, 256, 4)
    self.pool = nn.MaxPool2d(2, 2)
    self.fc1 = nn.Linear(256*6*6, 4096)
    self.fc2 = nn.Linear(4096, 1)

  def model(self, x):
    x = self.pool(F.relu(self.conv1(x)))
    x = self.pool(F.relu(self.conv2(x)))
    x = self.pool(F.relu(self.conv3(x)))
    x = F.relu(self.conv4(x))
    x = torch.flatten(x, 1)
    x = torch.sigmoid(self.fc1(x))
    return x

  def forward(self, x1, x2):
    x1_fv = self.model(x1)
    x2_fv = self.model(x2)
    ############################################################################
    ## TODO: Calculate the component-wise l1_distance between x1_fv and x2_fv ##
    # Fill out function and remove
    raise NotImplementedError("Student exercise: Calculate l1_distance")
    ############################################################################
    # Calculate L1 distance (as l1_distance) between x1_fv and x2_fv
    l1_distance = ...

    return self.fc2(l1_distance)
```

[*Click for solution*](https://github.com/NeuromatchAcademy/course-content-dl/tree/main//tutorials/W3D4_ContinualLearning/solutions/W3D4_Tutorial2_Solution_696c354e.py)

## Bonus Section 1.3: Verification using Siamese Networks

Let's train the siamese network we created on verification of images. We'll take pairs of images from the Omniglot dataset and train the network to identify if the pairs belong to the same class or not.

Let's first get the Omniglot dataset with pairs of images. A pair is labelled "1" if the images are of the same character, and "0" otherwise.

```python
# @title Download Omniglot dataset with pairs
import zipfile, os, requests

# original location: https://github.com/brendenlake/omniglot/tree/master/python
os.chdir(CWD)
print(f'Change dir: {os.getcwd()}')

dirname = 'data/omniglot-py/'
if not os.path.exists(dirname):
  os.makedirs(dirname)

fname = 'images_background.zip'
url = "https://osf.io/6hq9u/download"

if not os.path.exists(dirname + 'images_background'):
  print('Downlading the dataset...')
  r = requests.get(url, allow_redirects=True)
  with open(dirname + fname, 'wb') as fd:
    fd.write(r.content)
  with zipfile.ZipFile(dirname + fname, 'r') as zip_ref:
    zip_ref.extractall(dirname)
  print('Dataset is downloaded.')
else:
  print('Dataset has already been downloaded.')
```

```python
"""
Load train and validation datasets for training the Siamese Network
"""

train_dataset, val_dataset = get_train_val_datasets(background_dataset_size=10000, val_split=0.2, download=False)
```

```python
"""
Visualize a sample from the training dataset
"""

# Change this to visualize another sample from the dataset
sample_idx = 1
sample = train_dataset[sample_idx]

visualize_siamese_sample(sample)
```

```python
"""
Create dataloaders for the train and validation datasets
"""

# Change this for a different batch size
batch_size = 16

train_loader = DataLoader(
    dataset=train_dataset,
    batch_size=batch_size,
    num_workers=2,
    worker_init_fn=seed_worker,
    generator=g_seed
    )

val_loader = DataLoader(
    dataset=val_dataset,
    batch_size=batch_size,
    num_workers=2,
    worker_init_fn=seed_worker,
    generator=g_seed
    )
```

Now that we have the necessary dataloaders, let's train a siamese network!

```python
"""
Train the network on Omniglot
"""

siamese_net = ConvSiameseNet()

criterion = nn.BCEWithLogitsLoss()
optimizer = optim.Adam(siamese_net.parameters(), lr=0.0001)

# Change this to train for any number of epochs
num_epochs = 10

for epoch in range(num_epochs):
  print(f"\n Epoch {epoch + 1} / {num_epochs} ================================")

  train_siamese_network(siamese_net, criterion, optimizer, train_loader, DEVICE)
  evaluate_siamese_network(siamese_net, criterion, val_loader, DEVICE)
```

## Bonus Section 1.4: One-shot Classification with Siamese Networks

Now that we have a siamese network trained on verification, let's try to extend it to one-shot classification. How can we do this?

Let's first try to understand what the current network offers: Given two images, find out if they belong to the same class or not.

What does N-way one-shot classification ask for? Given one image from each of the N classes (the support set), find out which class does the query image belong to.

Take a moment to think about how the verification network can be extended to work in the one-shot setting.

Given a query image, compare it with different images in the support set using our siamese network. Pick the class that is most similar to the query image!

Let's get the test dataset where one sample corresponds to a query image and N support images!

```python
# @title Download the test dataset
import zipfile, os, requests

# original location: https://github.com/brendenlake/omniglot/tree/master/python

dirname = 'data/omniglot-py/'
if not os.path.exists(dirname):
  os.makedirs(dirname)

fname = 'images_evaluation.zip'
url = "https://osf.io/uq4gw/download"

if not os.path.exists(dirname + 'images_evaluation'):
  print('Downlading the dataset...')
  r = requests.get(url, allow_redirects=True)
  with open(dirname + fname, 'wb') as fd:
    fd.write(r.content)
  with zipfile.ZipFile(dirname + fname, 'r') as zip_ref:
    zip_ref.extractall(dirname)
  print('Dataset is downloaded.')
else:
  print('Dataset has already been downloaded.')
```

```python
"""
Load the test dataset for N-way One-shot classification
"""

# Change this to change the number of classes in the support set
n_ways = 5

test_dataset = get_test_dataset(n_ways=n_ways, download=True)
```

```python
"""
Visualize a sample from the test dataset
"""

# Change this to visualize another sample from the dataset
sample_idx = 1
sample = test_dataset[sample_idx]

visualize_one_shot_sample(sample)
```

We are in the endgame now. Let's look at how our siamese network performs on N-way One-shot classification!

```python
"""
Evaluate our Siamese Network on N-way One-shot classification
"""

batch_size = 16

test_loader = DataLoader(
    dataset=test_dataset,
    batch_size=batch_size,
    num_workers=2,
    worker_init_fn=seed_worker,
    generator=g_seed
    )

evaluate_one_shot(siamese_net, test_loader, DEVICE)
```

Congratulations! We successfully solved our first Meta-Learning problem. Feel free to run through this section again with different parameters and compare the differences!

## Bonus Section 1.5: Section Summary

In this section, we learnt the following (try to answer these questions to ensure that you understood the content fairly well):

* What is Meta-Learning and how is it used to solve various problems with Deep Learning?

* What is Few-shot Learning? We took a look at what is N-way K-shot classification using the Omniglot dataset and Convolutional Siamese Networks.

---
# Bonus 2: Continual and Life-Long Learning

```python
# @title Video 5: Continual and Life-Long Learning
from ipywidgets import widgets

out2 = widgets.Output()
with out2:
  from IPython.display import IFrame
  class BiliVideo(IFrame):
    def __init__(self, id, page=1, width=400, height=300, **kwargs):
      self.id=id
      src = "https://player.bilibili.com/player.html?bvid={0}&page={1}".format(id, page)
      super(BiliVideo, self).__init__(src, width, height, **kwargs)

  video = BiliVideo(id=f"BV1uX4y1c7Yh", width=854, height=480, fs=1)
  print("Video available at https://www.bilibili.com/video/{0}".format(video.id))
  display(video)

out1 = widgets.Output()
with out1:
  from IPython.display import YouTubeVideo
  video = YouTubeVideo(id=f"-lmY_lc75As", width=854, height=480, fs=1, rel=0)
  print("Video available at https://youtube.com/watch?v=" + video.id)
  display(video)

out = widgets.Tab([out1, out2])
out.set_title(0, 'Youtube')
out.set_title(1, 'Bilibili')

# add event to airtable
atform.add_event('Video 5: Continual and Life-Long Learning')

display(out)
```

## Bonus Think! 2: Why are Continual and Life-Long Learning tough?

Take a moment to think about the following:

1. You have had an extensive tutorial on Continual Learning. Can you quickly recall the different problems with Continual Learning and how these problems were solved?
2. What do you think makes Life-Long Learning tough to implement? How is this learning paradigm different from those that you have seen before?

---
# Bonus 3 - Feature Importance Tutorial

If you wish to understand how the accuracies are calculated in Section 3.3, continue reading!

Recall the problem setup:

There exist binary-valued random variables $X^{(1)}, X^{(2)}, X^{(3)}, Y \in {0,1}$ such that $X^{(1)}, X^{(2)}$, and $X^{(3)}$ are conditionally independent (given $Y$).

Let $P(Y=1) = 1/2$. Then the joint distribution of $X^{(1)}, X^{(2)}, X^{(3)}, Y$ is specified by the conditional probabilities $P(X_i=1 | Y=0)$ and $P(X_i=1 | Y=1)$, for $i=1,2,3$:

\begin{align}
P(X^{(1)}=1|Y=0)=0.10 &, P(X^{(1)}=1|Y=1)=0.0.90 \\
P(X^{(2)}=1|Y=0)=0.05 &, P(X^{(2)}=1|Y=1)=0.0.80 \\
P(X^{(3)}=1|Y=0)=0.01 &, P(X^{(1)}=1|Y=1)=0.0.71
\end{align}

## Define a feature importance metric

Here we'll use 0-1 loss to define feature importance. We can imagine using $X^{(1)}, X^{(2)}$, and $X^{(3)}$ to predict $Y$. $Y$ can only be 0 or 1, so your predictions must be 0 or 1 and will either be right or wrong (i.e, perfect for 0-1 loss).

## Calculate the individual feature importance (0-1 loss) for  $X^{(1)}$, $X^{(2)}$, and $X^{(3)}$

**Hint:** You can accomplish this by finding the probability that  X  and  Y  match. You're given $P(X^{(1)}=1|Y=0)$, but you need $P(X^{(1)}=0|Y=0)$ . Recall that $P(Y)=1/2$ and compute an equally weighted average of the two accuracies you can compute given the two possible values of $X^{(1)}$ .

**Solution:** If $P(X^{(1)} = 1|Y = 0)=0.10$, then $P(X^{(1)} = 0|Y = 0)=0.90$.

There's a symmetry we can take advantage of here: we know that 90% of the time $Y=1$ we have $X=1$, which implies that 90% of the time $X=1$ we have $Y=1$, which is a more natural way of thinking about prediction.

This means we predict correctly 90% of the time when $Y=0$, and we predict correctly 90% of the time when $Y=1$. Since $P(Y)=1/2$, we have 0-1 loss $L = 1-(0.5 \times 0.9 + 0.5 \times 0.9)=0.1$.

In total, we have:

\begin{align}
L_1 =& 1 - (0.5 \times 0.9 + 0.5 \times 0.9) = 0.1 \\
L_2 =& 1 - (0.5 \times 0.95 + 0.5 \times 0.8) = 0.125 \\
L_3 =& 1 - (0.5 \times 0.99 + 0.5 \times 0.71) = 0.15.
\end{align}

**Check:** If you did not observe that feature $X^{(1)}$ is the most important feature by virtue of having the smallest 0-1 loss (0.1), check your calculations before proceeding.

### Calculate the importance of feature pairs (0-1 loss) for $\{X^{(1)}, X^{(2)}\}$, $\{X^{(1)}, X^{(3)}\}$, and $\{X^{(2)}, X^{(3)}\}$

### Let's start with $\{X^{(1)}, X^{(2)}\}$

Working out these calculations is a bit tedious, so we'll provide various tables and descriptions of their origins.

First, we compute conditional PMFs $P\left( X^{(1)}, X^{(2)}∣Y \right)$ by multiplying the provided conditionally independent probabilities together.

**Conditional PMFs:**

\begin{matrix}
P(X_1,X_2|Y=0) && X_2=0 && X_2=1 \\
\hline
X_1=0 && 0.855 && 0.045 \\
X_1=1 && 0.095 && 0.005
\end{matrix}

\begin{matrix}
P(X_1,X_2|Y=1) && X_2=0 && X_2=1 \\
\hline
X_1=0 && 0.02 && 0.08 \\
X_1=1 && 0.18 && 0.72
\end{matrix}

We can then obtain a marginal PMF for $\{X^{(1)}, X^{(2)}\}$ by element-wise averaging these two 2x2 tables because $P(Y)=1/2$.

**Marginal PMF:**

\begin{matrix}
P(X_1,X_2) && X_2=0 && X_2=1 \\
\hline
X_1=0 && 0.4375 && 0.0625 \\
X_1=1 && 0.1375 && 0.3625
\end{matrix}

We now use the conditional PMFs to evaluate the conditional probability expressions we care about: $P\left( Y∣X^{(1)}, X^{(2)} \right)$.

**Conditional Probabilities: $P \left( Y=\{0, 1\}|X_1,X_2 \right)$**

Using the $\frac{P(X_1,X_2 | Y=0)}{P(X_1,X_2|Y=0) + P(X_1,X_2|Y=1)}$, we obtain the following matrix:

\begin{matrix}
P(Y=0|X_1,X_2) && X_2=0 && X_2=1 \\
\hline
X_1=0 && 0.977143 && 0.36 \\
X_1=1 && 0.345455 && 0.006897
\end{matrix}

Using the $\frac{P(X_1, X_2 | Y=1)}{P(X_1,X_2|Y=0) + P(X_1,X_2|Y=1)}$, we obtain the following matrix:

\begin{matrix}
P(Y=1|X_1,X_2) && X_2=0 && X_2=1 \\
\hline
X_1=0 && 0.022857 && 0.640000 \\
X_1=1 && 0.654545 && 0.993103
\end{matrix}

We now implement a 0-1 classifier that predicts $\hat{Y}=0$ or $\hat{Y}=1$ based on the element-wise maximum in these two tables. For example, when $X^{(1)}=0$ and $X^{(2)}=0$, there's a 0.977 probability that $Y=0$ and a 0.023 probability that $Y=1$. We would of course predict $\hat{Y}=1$ , and we expect to be correct 97.7% of the time.

We now know the expected probability of success for our classifier for each $\{X^{(1)}, X^{(2)}\}$ combination.

**Maximum Conditional Probability:**

\begin{matrix}
P(\text{Correct}|X_1,X_2) && X_2=0 && X_2=1 \\
\hline
X_1=0 && 0.977143 && 0.640000 \\
X_1=1 && 0.654545 && 0.993103
\end{matrix}

Only one task remains for you to complete: compute the expected 0-1 loss for this classifier.

**Hint:** We know how likely we are to be correct given some $\{X(1), X(2)\}$ combination. We also know how likely those combinations are to appear.

**Solution:**

```python
marginal = np.array([[0.4375, 0.0625],
                     [0.1375, 0.3625]])
max_cond_prob = np.array([[0.977, 0.64],
                          [0.6545, 0.993]])
# We need the sum-product of these two arrays.
# (1) element-wise multiplication to produce one array.
# (2) sum the elements of the resulting array
accuracy = np.sum(np.multiply(marginal, max_cond_prob))
print(accuracy)
```

```python
# 0-1 loss is 1 - accuracy.
print(1 - accuracy)
```

**Check:** You should have gotten 0-1 loss for $\{X^{(1)}, X^{(2)}\}$ to be 0.0826.

### Let's move on to  $\{X^{(2)}, X^{(3)}\}$

Our claim was that the most important single feature, already found to be $\{X^{(1)}$\}, is not in the most important pair of features, so we just need to show that the 0-1 loss for $\{X^{(2)}, X^{(3)}\}$ is less than 0.0826.

First, we compute conditional PMFs $P \left( X^{(2)}, X^{(3)}∣Y \right)$ by multiplying conditionally independent probabilities together.

**Conditional PMFs:**

\begin{matrix}
P(X_2,X_3|Y=0) && X_3=0 && X_3=1 \\
\hline
X_2=0 && 0.9405 && 0.0095 \\
X_2=1 && 0.0495 && 0.0005
\end{matrix}

\begin{matrix}
P(X_2,X_3|Y=1) && X_3=0 && X_3=1 \\
\hline
X_2=0 && 0.058 && 0.142 \\
X_2=1 && 0.232 && 0.568
\end{matrix}

We can then obtain a marginal PMF for $\{X^{(2)}, X^{(3)}\}$ by element-wise averaging these two 2x2 tables because $P(Y)=1/2$.

**Marginal PMF:**

\begin{matrix}
P(X_2,X_3) && X_3=0 && X_3=1 \\
\hline
X_2=0 && 0.49925 && 0.07575 \\
X_2=1 && 0.14075 && 0.28425
\end{matrix}

We now use the conditional PMFs to evaluate the conditional probability expressions we care about: $P \left( Y∣X^{(2)}, X^{(3)} \right)$.

**Conditional Probabilities: $P \left( Y=\{0, 1\}|X_2, X_3 \right)$**

Using the $\frac{P(X_2,X_3 | Y=0)}{P(X_2, X_3|Y=0) + P(X_2, X_3|Y=1)}$, we obtain the following matrix:

\begin{matrix}
P(Y=0|X_2,X_3) && X_3=0 && X_3=1 \\
\hline
X_2=0 && 0.941912869 && 0.062706271 \\
X_2=1 && 0.175843694 && 0.000879507
\end{matrix}

Using the $\frac{P(X_2,X_3 | Y=1)}{P(X_2, X_3|Y=0) + P(X_2, X_3|Y=1)}$, we obtain the following matrix:

\begin{matrix}
P(Y=0|X_2,X_3) && X_3=0 && X_3=1 \\
\hline
X_2=0 && 0.058087131 && 0.937293729 \\
X_2=1 && 0.824156306 && 0.999120493
\end{matrix}

We now implement a 0-1 classifier that predicts $\hat{Y}=0$ or $\hat{Y}=1$ based on the element-wise maximum in these two tables. For example, when $X^{(2)}=0$ and $X^{(3)}=0$, there's a 0.942 probability that $Y=0$ and a 0.058 probability that $Y=1$. We would of course predict $\hat{Y}=1$ , and we expect to be correct 94.2% of the time.

We now know the expected probability of success for our classifier for each $\{X^{(2)}, X^{(3)}\}$ combination.

**Maximum Conditional Probability:**

\begin{matrix}
P(\text{Correct}|X_2,X_3) && X_3=0 && X_3=1 \\
\hline
X_2=0 && 0.941912869 && 0.937293729 \\
X_2=1 && 0.824156306 && 0.999120493
\end{matrix}

Only one task remains for you to complete: compute the expected 0-1 loss for this classifier.

**Hint:** We know how likely we are to be correct given some $\{X(2), X(3)\}$ combination. We also know how likely those combinations are to appear.

**Solution:**

```python
marginal = np.array([[0.49925, 0.07575],
                     [0.14075, 0.28425]])
max_cond_prob = np.array([[0.941912869, 0.937293729],
                          [0.824156306,0.999120493]])
# We need the sum-product of these two arrays.
# (1) element-wise multiplication to produce one array.
# (2) sum the elements of the resulting array
accuracy = np.sum(np.multiply(marginal, max_cond_prob))
print(accuracy)
```

```python
# 0-1 loss is 1 - accuracy.
print(1 - accuracy)
```

**Check:** You should have gotten 0-1 loss for $\{X^{(2)}, X^{(3)}\}$ to be 0.05875.

## Conclusion

We demonstrated that our intuition could be misleading - it's not always the case that a feature that ranks highly among individual features would necessarily warrant being included at all when larger sets of features are considered.

Something to think about: What does this tell you about stepwise feature selection approaches?


$\require{mhchem}$
$\require{physics}$

$\newcommand{\Ce}{\ce{e}}$
$\newcommand{\Car}{\ce{Ar}}$
$\newcommand{\Carp}{\ce{Ar+}}$
$\newcommand{\Carm}{\ce{Ar}^{*}}$
$\newcommand{\Csihp}[1]{\ce{SiH#1}}$
$\newcommand{\Ceps}{\varepsilon}$

$\newcommand{\Ne}{n_{\Ce}}$
$\newcommand{\Nar}{n_{\ce{Ar}}}$
$\newcommand{\Narp}{n_{\ce{Ar+}}}$
$\newcommand{\Narm}{n_{\ce{Ar}^{*}}}$
$\newcommand{\Nsihp}[1]{n_{\ce{SiH#1}}}$
$\newcommand{\Neps}{n_{\varepsilon}}$

$\newcommand{\Kel}{k_{\text{el}}}$
$\newcommand{\Ki}{k_{\text{i}}}$
$\newcommand{\Kex}{k_{\text{ex}}}$
$\newcommand{\Kiarm}{k_{\text{i},\Carm}}$
$\newcommand{\Kelsihf}{k_{\text{el},\Csihp{4}}}$
$\newcommand{\Kdisihf}{k_{\text{di},\Csihp{4}}}$
$\newcommand{\Kdsih}[1]{k_{\text{d},\Csihp{#1}}}$
$\newcommand{\Kisiht}{k_{\text{i},\Csihp{3}}}$
$\newcommand{\Kv}[1]{k_{\text{v#1}}}$
$\newcommand{\Kn}[1]{k_{\text{#1}}}$

$\newcommand{\Eel}{\varepsilon_{\text{el}}}$
$\newcommand{\Ei}{\varepsilon_{\text{i}}}$
$\newcommand{\Eex}{\varepsilon_{\text{ex}}}$
$\newcommand{\Eiarm}{\varepsilon_{\text{i},\Carp}}$
$\newcommand{\Eelsihf}{\varepsilon_{\text{el},\Csihp{4}}}$
$\newcommand{\Edisihf}{\varepsilon_{\text{di},\Csihp{4}}}$
$\newcommand{\Edsih}[1]{\varepsilon_{\text{d},\Csihp{#1}}}$
$\newcommand{\Eisiht}{\varepsilon_{\text{i},\Csihp{3}}}$
$\newcommand{\Ev}[1]{\varepsilon_{\text{v#1}}}$
$\newcommand{\En}[1]{\varepsilon_{\text{#1}}}$
$\newcommand{\dv}[2]{\frac{\text{d}#1}{\text{d}#2}}$

   Copyright 2019 Benjamin Santos

   Licensed under the Apache License, Version 2.0 (the "License");
   you may not use this file except in compliance with the License.
   You may obtain a copy of the License at

       http://www.apache.org/licenses/LICENSE-2.0

   Unless required by applicable law or agreed to in writing, software
   distributed under the License is distributed on an "AS IS" BASIS,
   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
   See the License for the specific language governing permissions and
   limitations under the License.

If you find this notebook useful, please cite as: Santos, B., Cacot, L., Boucher, C. & Vidal, F. Electrostatic enhancement factor for the coagulation of silicon nanoparticles in low-temperature plasmas. Plasma Sources Sci. Technol. 28, 045002 (2019).

```@article{santos_electrostatic_2019,
	title = {Electrostatic enhancement factor for the coagulation of silicon nanoparticles in low-temperature plasmas},
	volume = {28},
	issn = {0963-0252},
	url = {https://doi.org/10.1088%2F1361-6595%2Fab0a2b},
	doi = {10.1088/1361-6595/ab0a2b},
	number = {4},
	urldate = {2019-06-01},
	journal = {Plasma Sources Sci. Technol.},
	author = {Santos, Benjamin and Cacot, Laura and Boucher, Claude and Vidal, François},
	month = apr,
	year = {2019},
	pages = {045002},
}```

```python
%load_ext autoreload
%autoreload 2
```

```python
import numpy as np
import scipy.constants as const
from scipy.integrate import solve_ivp
```

```python
import os
import sys
module_path = os.path.abspath(os.path.join('..'))
if module_path not in sys.path:
    sys.path.append(module_path)
os.chdir('../')
```

```python
import trazar as tzr
import plazma as pz
```

# Plasma model

We considered the following reactions,
$$\begin{equation*}
  \begin{array}{llll}
    \text{R1}&\Kel&\text{elastic}& \ce{Ar + e -> Ar + e}\\
    \text{R2}&\Ki&\text{ionization}& \ce{Ar + e -> Ar^+ + 2e}\\
    \text{R3}&\Kex&\text{excitation}& \ce{Ar + e -> Ar}^*\ce{+ e}\\
    \text{R4}&\Kiarm&\text{ionization}& \ce{Ar}^*\ce{ + e -> Ar^+ + 2e}\\
    \text{R5}&\Kelsihf&\text{elastic}& \ce{SiH_4 + e -> SiH_4 + e}\\
    \text{R6}&\Kdisihf&\text{diss. ionization}&\ce{SiH_4 + e -> SiH^+_3 + H + 2e}\\
    \text{R7}&\Kdsih{3}&\text{dissociation}&\ce{SiH_4 + e -> SiH_3 + H + e}\\
    \text{R8}&\Kdsih{2}&\text{dissociation}&\ce{SiH_4 + e -> SiH_2 + 2H + e}\\
    \text{R9}&\Kisiht&\text{ionization}&\ce{SiH_3 + e -> SiH^+_3 + 2e}\\
    \text{R10}&\Kv{13}&\text{vibrational}&\ce{SiH_4 + e -> SiH^{v13}_4 + e}\\
    \text{R11}&\Kv{24}&\text{vibrational}&\ce{SiH_4 + e -> SiH^{v24}_4 + e}\\
    \text{R12}&\Kn{12}&\text{metastable}& \ce{SiH4 + Ar^{\star} -> SiH3 + H + Ar}\\
    \text{R13}&\Kn{13}&\text{metastable}& \ce{SiH4 + Ar^{\star} -> SiH2 + 2H + Ar}\\
    \text{R14}&\Kn{14}&\text{metastable}& \ce{SiH3 + Ar^{\star} -> SiH2 + H + Ar}\\
    \text{R15}&\Kn{15}&\text{metastable}& \ce{SiH2 + Ar^{\star} -> SiH + H + Ar}
  \end{array}
\end{equation*}$$
with coefficient rates,

```python
rates_dict = dict()
```

```python
rates_dict["R1:kel"] = ("a1expb", 1.8560282921521678e-13, 0.1633129254054345, 0.0)
rates_dict["R2:ki"] = ("arrhenius", 7.41485579e-14, 15.8, 5.00954185e-3)
rates_dict["R3:kex"] = ("arrhenius", 3.364506180166679e-14, 11.5, 3.7596917572476254e-09)
rates_dict["R4:kiarm"] = ("arrhenius", 1.911868293754553e-13, 4.2, 2.6193020484482345e-09)
rates_dict["R5:kelsih4"] = ("a1expb", 2.663032076381496e-13, 0.27729017757606, 0.0)
rates_dict["R6:kdisih4"] = ("arrhenius", 3.6864532111053647e-14, 12.3, 7.585426699230944e-09)
rates_dict["R7:kdsih3"] = ("arrhenius", 1.060628946494521e-13, 8.4, 0.07797295284736888)
rates_dict["R8:kdsih2"] = ("arrhenius", 8.999280746645033e-14, 7.7, 9.918880635931725e-10)
rates_dict["R9:kisih3"] = ("arrhenius", 4.6999933119522564e-14, 8.0, 0.1625740097963909)
rates_dict["R10:kv13"] = ("arrhenius",  5.2570315025458126e-15, 0.27, 7.675878736753314e-09)
rates_dict["R11:kv24"] = ("arrhenius", 9.646752429918597e-15, 0.113, 1.5565618639018595e-08)
rates_dict["R12:k12"] = ("constant", 1.4e-16, 0.0, 0.0)
rates_dict["R13:k13"] = ("constant", 2.59e-16, 0.0, 0.0)
rates_dict["R14:k14"] = ("constant", 9.96e-17, 0.0, 0.0)
rates_dict["R15:k15"] = ("constant", 9.96e-17, 0.0,0.0)
```

```python
rmap = pz.RatesMap(rates_dict)
```

```python
rates_map = rmap.get_ratesmap()
```

```python
energy = np.linspace(0.01, 25.0, 500)
```

```python
rmap.plot_rates(energy)
```

```python
plasmasystem = pz.PlasmaSystem(4e-2,
                               6e-2,
                               300.0,
                               300.0,
                               0.1,
                               29.0/30.0,
                               6.6335209e-26,
                               5.3150534e-26,
                               1./const.elementary_charge,
                               1.0)
```

```python
print(plasmasystem)
```

    System(length=0.04, radius=0.06, temperature=300.0, ion_temperature=300.0, pressure_torr=0.1, arsih4_ratio=0.9666666666666667, armass=6.6335209e-26, sih4mass=5.3150534e-26, power=6.241509125883258e+18, with_tunnel=1.0)

We solve for the species densities and the energy density $\Neps$:
$$\begin{equation}
 \Ce, \Carp, \Carm , \Csihp{3+}, \Csihp{3}, \Csihp{2}
\end{equation}$$

using the following set of equations for these species:
$$\begin{align}
%\Kel, \Ki, \Kex, \Kiarm, \Kelsihf, \Kdisihf, \Kdsih{3}, \Kdsih{2}, \Kisiht, \Kv{13}, \Kv{24}, \Kn{12}, \Kn{13}
 \dv{\Ne}{t} &= \Ki\Ne\Nar + \Kiarm\Ne\Narm + \Kdisihf\Ne\Nsihp{4} + \Kisiht\Ne\Nsihp{3}-\frac{A}{V}\Gamma_{\Carp}\Narp
 -\frac{A}{V}\Gamma_{\Csihp{3+}}\Nsihp{3+} - \sum_{ik}L^e_{ik}\Ne,\\
 \dv{\Narm}{t} &= \Kex \Nar\Ne - \Kiarm\Ne\Narm -\Kn{12}\Narm\Nsihp{4}-\Kn{13}\Narm\Nsihp{4}
 -\Kn{14}\Narm\Nsihp{3}-\Kn{15}\Narm\Nsihp{2}-\frac{A}{V}\Gamma_{\Carm}\Narm,\\
 \dv{\Csihp{3+}}{t} &= \Kdisihf\Ne\Nsihp{4}-\Kisiht\Ne\Nsihp{3}-\frac{A}{V}\Gamma_{\Csihp{3+}}\Nsihp{3+},\\
 \dv{\Csihp{3}}{t} &= \Kdsih{3}\Ne\Nsihp{4} -\Kisiht\Ne\Nsihp{3} +\Kn{12}\Narm\Nsihp{4}-\Kn{14}\Narm\Nsihp{3}
 -\frac{A}{V}\Gamma_{\Csihp{3}}\Nsihp{3},\\
 \dv{\Csihp{2}}{t} &= \Kdsih{2}\Ne\Nsihp{4} +\Kn{13}\Narm\Nsihp{4}+\Kn{14}\Narm\Nsihp{3}-\Kn{15}\Narm\Nsihp{2}
 -\frac{A}{V}\Gamma_{\Csihp{2}}\Nsihp{2},\\
\end{align}$$
along with the equation for the energy density:
$$\begin{align}
 \dv{\Neps}{t} &= \frac{P}{V} - \Ei\Ki\Nar\Ne - \Eex\Kex\Nar\Ne - \Eiarm\Kiarm\Narm\Ne
 -\frac{5}{3}\frac{A}{V}v_{\text{Bohm},\Carp}\Neps- e V_{\text{sheath}}\frac{A}{V}v_{\text{Bohm},\Carp}\Ne\nonumber\\
 &-3\frac{m_e}{m_{\Car}}\Kel\Neps\Nar -3\frac{m_e}{m_{\Csihp{4}}}\Kelsihf\Neps\Nsihp{4}
 - \Eisiht\Kisiht\Ne\Nsihp{3} - \Edisihf\Kdisihf\Ne\Nsihp{4} - \Edsih{3}\Kdsih{3}\Ne\Nsihp{4}\nonumber\\
 &- \Edsih{2}\Kdsih{2}\Ne\Nsihp{4} - \Ev{13}\Kv{13}\Ne\Nsihp{4}  - \Ev{24}\Kv{24}\Ne\Nsihp{4}
 - \sum_{ik}\Phi_{ik}L^e_{ik}\Ne,
\end{align}$$
and the quasi-neutrality condition:
$$\begin{equation}
 \Nsihp{3} = \Ne - \Narp + N^Q_\text{NP}.
\end{equation}$$

Here A and V are the area and volume of the reactor, $\Gamma$ is the flux,
$L^e_{\text{NP}}$ is the loss of electrons to nanoparticles (in $s^{-1}$ see below)
$P$ is the power density,
$\Phi_{ik}$ the surface potential of nanoparticle $i$ with charge $k$ in $eV$, and $N^Q_\text{NP}$
is the net charge on nanoparticles.

```python
pc = pz.PlasmaChem(rates_map, plasmasystem)
sys = pc.get_system()
```

```python
%%time
sol = solve_ivp(sys, [0, 10e-3], [1.0,1.0,1.0,1.0,1.0,1.0,1.0], method='BDF', dense_output=False)
```

    CPU times: user 128 ms, sys: 226 µs, total: 128 ms
    Wall time: 128 ms

```python
xlabel = "Time (s)"
ylabel = r'Density (m$^{-3}$)'
labels = ['e', 'Ar+', 'Ar*', 'SiH3+', 'SiH3', 'SiH2']
densities = [sol.y[0], sol.y[1], sol.y[2], sol.y[3], sol.y[4], sol.y[5]]
```

```python
np.sum(sol.y[0]-sol.y[1]-sol.y[3])
```

    -324705.6120838639

```python
tzr.plot_plain(sol.t, densities, title="Densities",
                       axislabel=[xlabel, ylabel],
                       logx=False, logy=True, labels=labels, ylim=[1e10, 1e18], savename='plasma-densities.eps')
```

```python
tzr.plot_plain(sol.t, sol.y[-1]/sol.y[0], title="Mean electron energy",
                       axislabel=['Time (s)', 'Energy (ev)'],
                       logx=False, logy=False, labels=labels, ylim=[0, 3], savename='energy.eps')
```

```python

```