# Deep Learning, uncertainty and probabilistic models.

> **Certainty** is perfect knowledge that has total security from error, or the mental state of being without doubt. (*Wikipedia*)

> **Uncertainty** has been called *an unintelligible expression without a straightforward description*. It describes a situation involving insecurity and/or unknown information. It applies to predictions of future events, to physical measurements that are already made, or to the unknown. (*Wikipedia*)

A common understanding of the term **uncertainty** in scientific works is that of a measure that reflects the amount of dispersion of a random variable. In other words, it is a scalar measure of how "random" a random variable is. But this a reduction of the term.

> There is **no single formula for uncertainty** because there are many different ways to measure dispersion: standard deviation, variance, and entropy are all appropriate measures. However, it's important to keep in mind that a single scalar number cannot paint a full picture of "randomness'', as that would require communicating the entire random variable itself!  *Eric Jang*.

When working with predictive systems, measuring uncertainty is **important for dealing with the risk associated to decisions**. If we can measure risk, we can define policies related to making use of predictions.

Regarding uncertainty, we can be in several **states**:

+ Complete certainty. This is the case of macroscopic mechanics, where given an input there is no uncertainty about the output.
+ Uncertainty with risk. This is the case of a game with an uncertain output, but where we exactly know the probability distribution over outcomes.
+ Fully reducible uncertainty. This is the case of a game where by gathering sufficient data and/or using the right model we can get complete certainty.
+ Partially reducible uncertainty. This is the case of games where we have no full knowledge about the probability distribution over outcomes. By gathering sufficient data we can know the probability distribution over outcomes.
+ Etc.

Uncertainty in predictive systems $y = f_w(x)$ can be **caused** by several factors (see next). Different  types  of  uncertainty  must  be  measured in different ways.

## Epistemic uncertainty

Epistemic uncertainty corresponds to the uncertainty originated by the lack of knowledge about the model.  It can be explained as to which extent is our model able to describe the distribution that generated the data.

In this case we can talk of two different types of uncertainties caused by whether the model has been **trained with enough data**, so it’s been able to learn the full distributionof the data, or whether the **expressiveness of the model is able to capture the complexity of the distribution**.

When using a expressive enough model, this type of uncertainty *can be reduced* by providing more samples during the training phase.

##  Aleatoric uncertainty

 In this case the uncertainty measured belongs to the data.   This uncertainty is inherent to the data and *can’t be reduced* by adding more data to the training process. Its cause can be the lack of information in $x$ to determine $y$ or the lack of precision when measuring $x$ or $y$.

**Aleatoric uncertainty propagates from the inputs to the model predictions**. Consider a simple model $y=5x$, which takes in normally-distributed input $x  \sim N(0,1)$. In this case, $y \sim N(0,5)$, so the aleatoric uncertainty of the predictive distribution can be described by $\sigma=5$.

There are two types of aleatoric uncertainty:

+ **Homoscedastic**:  Uncertainty remains constant for all the data. For example:   $y \sim N(0,5)$.
+ **Heteroscedastic**: Uncertainty depends on the input. For example:   $y \sim N(0,f(x))$.

## Out of Distribution (OoD)

Determining whether inputs are "valid'' is a serious problem for deploying ML in the wild, and is known as the Out of Distribution (OoD) problem.

There are two ways to handle OoD inputs for a machine learning model: 1) Model the bad inputs and detect them before we even put them through the model 2) Let the "weirdness'' of model predictions imply to us that the input was probably malformed.

Modeling bad inputs is a difficult task. Training a discriminator is not completely robust because it can give arbitrary predictions for an input that lies in neither distribution. Instead of a discriminator, we could build a density model of the in-distribution data, such as a kernel density estimator.

The second approach to OoD detection involves using the predictive (epistemic) uncertainty of the task model to tell us when inputs are OoD. Ideally, malformed inputs to a model ought to generate "weird'' predictive distribution $p(y|x)$. For instance, it has been showed that the maximum softmax probability (the predicted class) for OoD inputs tends to be lower than that of in-distribution inputs.

## Example

Uncertainty can be understood from a **simple machine learning model** that attempts to predict daily rainfall from a sequence of barometer readings. Aleatoric uncertainty is irreducible randomness that arises from the data collection process. Epistemic uncertainty reflects confidence that our model is making the correct predictions. Finally, out-of-distribution errors arise when the model sees an input that differs from its training data (e.g. temperature of the sun, other anomalies). *Eric Jang*.

## Calibration

Just because a model is able to output a "confidence interval" for a prediction doesn't mean that the confidence interval actually reflects the actual probabilities of outcomes in reality!

Suppose our rainfall model tells us that there will be $N(4,1)$ inches of rain today. If our model is **calibrated**, then if we were to repeat this experiment over and over again under identical conditions (possibly re-training the model each time), we really would observe empirical rainfall to be distributed exactly $N(4,1)$.

Machine Learning models mostly optimize for test accuracy or some fitness function. Researchers are not performing model selection by deploying the model in repeated identical experiments and measuring calibration error, so unsurprisingly, this **models tend to be poorly calibrated**.

## Uncertainty and Neural Networks

A trained neural network $f$ can be viewed as the instantiation of a specific probabilistic model $p(y|x,D)$.

For classification, $y$ is a set of classes and $p(y|x,D)$ is a **categorical distribution**.

For regression, $y$ is a continuous variable and $p(y|x,D)$ can be modelled with a **Gaussian (or Laplacian) distribution**.

Our training goal  can be  to  **find  the  most probable network instance** (represented by parameters $w$)  that  generated the outputs:

$$
p(y | x, D) = \int_w p(y | x, w)p(w|D)dw
$$

We can observe that the distribution of the output depends on two terms: one that depends on the application of the **model to the input data**, and a second one that is measuring **how the model may  vary  depending  on  the  training  data**.

From this definition we can say that the first term is modeling the **aleatoric uncertainty**, as it is measuring  how  the  output  is distributed when the input is $x$,  and the second term is modeling the **epistemic uncertainty** as it is measuring the uncertainty induced by the parameters of the model.

## Epistemic Uncertainty

In  the  case  of  the epistemic uncertainty, let’s assume that we want to train a model with parameters $w$ that produces an output $y = f_w(x)$.

To capture epistemic uncertainty in a neural network we will adopt a Bayesian point of view and we will put a prior distribution over its weights, for example a Gaussian prior distribution: $w ∼ \mathcal{N} (0, I)$. Such a model is referred to as a  **Bayesian neural network**. Bayesian neural networks replace the deterministic network’s weight parameters with distributions over these parameters.

In the case of regression, we suppose the model likelihood is $p(y | x,w) ∼ \mathcal{N} (f_w(x), \sigma^2)$.

For classification tasks, we assume a softmax likelihood:

$$
p(y = d | x, w) = \frac{\exp(f^d_w(x))}{\sum_{d'}\exp(f^{d'}_w(x))}
$$

Given a dataset $D = (X,Y)$  we then look for the posterior distribution over the space of parameters by invoking Bayes’ theorem:

$$
p(w|X,Y) = \frac{p(Y|X,w) p(w)}{p(Y|X)}
$$

The true posterior $p(w|X, Y)$ cannot usually be evaluated analytically, but we can use indirect methods such as variational inference or MC-dropout.

This distribution captures the most probable function parameters given our observed data. With it we can predict an output for a new input point $x$ by integrating:

$$
p(y|x,X,Y) = \int p(y|x,w)p(w|X,Y)dw
$$

This integral can be computed by Monte Carlos method:

$$
\mathop{\mathbb{E}}(y|x, X,Y) \approx \frac{1}{M}\sum_{j=1}^{M} f_{{w_{j}}}(x)
$$

Finaly we can calculate the corresponding variance:
$$
\mathop{\mathbb{Var}} (y) \approx \frac{1}{M}\sum_{j=1}^{M} f_{w_{j}}(x)^2 - \mathop{\mathbb{E}}(y|x, X,Y)^2
$$

Which can be used as a measure of the **epistemic uncertainty**.

## Aleatoric Uncertainty

For obtaining the heteroscedastic uncertainty, we follow the same approach as in the epistemic case: we consider that we have a **fixed model**  $f_w(x)$ and we want to observe the variability of the term $p(y|f_w(x))$.

In **regression tasks**, , we can assume that $y \sim \mathcal{N}(f_{{w}}(x), \sigma(x)^2)$, where $f_{w}(x)$ is the predicted value of $x$ and $\sigma_(x)$ is the unknown variance of this value. How do we estimate this value?

In the same way we use a deep learning model to estimate $f_w(x)$, we can use a deep learning model to estimate variance: $\sigma_w(x)$, and then $y \sim \mathcal{N}(f_{{w}}(x), \sigma_w(x)^2)$.

Applying this approximation to the log-likelihood adds an additional term to the loss that depends on $\sigma_w(x)$:

$$
\mathcal L = - \frac{1}{N} \sum_{i=1}^N \log p(y|f_w(x_i))
$$

$$
= \frac{1}{2 \sigma(x)^2} ( y - f_w(x))^2 + \frac{1}{2} \log \sigma_w(x)^2
$$

We can easily optimiza a model that outputs $(f_{{w}}(x), \sigma_w(x))$ by using the reparametrization trick in the last layer of the network.

In **classification tasks**, this approximation is not as straightforward as in regression, as for classification there is already an uncertainty component due to the distribution of the probabilities when applying the softmax to the logits.

In this scenario, we can apply the same assumption but to the logits space instead of the output directly:

$$\begin{split}
logits \sim \mathcal{N}(f_{w}(x), diag(\sigma_w(x)^2))  \\
p = softmax(logits) \\
y \sim Categorical(p)
\end{split}
$$

Here we can apply the reparametrization trick for computing the logits, $u$:

$$
\begin{split}
u \sim \mathcal{N}(f_{w}(x), diag(\sigma_w(x)^2))  \\
u = f_{w}(x) + \sqrt{diag(\sigma_w(x)^2)}* \epsilon\\
u = f_{w}(x) + diag(\sigma_w(x))* \epsilon, \epsilon\sim\mathcal{N}(0,1)
\end{split}
$$

And then apply Monte Carlo sampling to obtain the expectation of the probability:

$$
\mathop{\mathbb{E}}[p] = \frac{1}{M}\sum^{M}_{m=1}softmax(u^{m})
$$

When applied to a crossentropy loss we have that:

$$
\begin{align}
\ell(W) & = -\frac{1} {C}\sum^{C}_{c=1}y_{i,c}\log{(p_{i,c})}  \\
& = -\frac{1}{C}\sum^{C}_{c=1}y_{i,c}\log{\frac{1}{M}\sum^{M}_{m=1}softmax(u^{m})}\\
& = -\frac{1}{C}\sum^{C}_{c=1}y_{i,c}\log{\frac{1}{M}\sum^{M}_{m=1}\frac{\exp{u^{m}_{i,c,m}}}{\sum^{C}_{c{}'=1}\exp{u^{m}_{i,c{}',m}}}} \\
& =  -\frac{1}{C}\sum^{C}_{c=1}y_{i,c}\log{\sum^{M}_{m=1}\exp({u^{m}_{i,c,m}}} - \log{\sum^{C}_{c'=1}\exp{u^{m}_{i,c',m}})}-\log{M} \\
\end{align}
$$

Where $C$ is the number of classes and $M$ is the number of Monte Carlo samples taken.

Once trained, the sigmas can be used to obtain a measure of the aleatoric uncertainty associated to the input by calculating the mean variance of the logits:

$$
\begin{align}
\mathop{\mathbb{U}} & = \frac{1}{C}\sum^{C}_{c=1}var[p_{c}] \\
&= \frac{1}{C}\sum^{C}_{c=1}var(softmax(u^{m}_{c})), m \in \left \{1,M\right \}\end{align}
$$

## Total uncertainty

Epistemic and aleatoric uncertinty can be combined in a model. Then, the predictive uncertainty for $y$ can be approaximated by:

$$
Var(y) \approx \frac{1}{T} \sum_{t=1}^{T} y_t^2 - (\frac{1}{T} \sum_{t=1}^{T} y)^2 + \frac{1}{T} \sum_{t=1}^{T} \sigma_t(x)^2
$$

with ${y_t,\sigma_t(x)}$ a set of $T$ sampled outputs: $y_t, \sigma_t (x) = f_{w_t} (x)$ for random weights $w_t \sim p(w)$.

# Predicting probability distributions

#### Regression with a pre-defined homoscedastic noise model

A network can now be trained with a Gaussian negative log likelihood function (`neg_log_likelihood`) as loss function assuming a **fixed standard deviation** (`noise`).

This is equivalent to consider the following loss function:

$$
LogLoss = \sum_i \frac{(y_i - f(x_i))^2}{2 \sigma^2}+\frac{1}{2} \log \sigma^2
$$

where the model predicts a mean $f(x_i)$.

```
import tensorflow
import numpy as np
import matplotlib.pyplot as plt

%matplotlib inline

def f(x, sigma):
    epsilon = np.random.randn(*x.shape) * sigma
    return 10 * np.sin(2 * np.pi * (x)) + epsilon

train_size = 320
noise = 1.0

plt.figure(figsize=(8,4))
X = np.linspace(-0.8, 0.8, train_size).reshape(-1, 1)
y = f(X, sigma=noise)
y_true = f(X, sigma=0.0)

plt.scatter(X, y, marker='+', label='Training data')
plt.plot(X, y_true, label='Truth', color='r')
plt.title('Noisy training data and ground truth')
plt.legend();

print(X[0],y[0],y_true[0])
```

```
from tensorflow.keras import backend as K
from tensorflow.keras import activations, initializers
from tensorflow.keras.layers import Input, Dense
from tensorflow.keras.models import Model
from tensorflow.keras import callbacks, optimizers
import tensorflow as tf
import tensorflow_probability as tfp

x_in = Input(shape=(1,))
x    = Dense(20, activation='relu')(x_in)
x    = Dense(20, activation='relu')(x)
x    = Dense(1)(x)

model = Model(x_in, x)
model.summary()
```

    WARNING: The TensorFlow contrib module will not be included in TensorFlow 2.0.
    For more information, please see:
      * https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
      * https://github.com/tensorflow/addons
    If you depend on functionality not listed there, please file an issue.

    WARNING:tensorflow:From /usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/resource_variable_ops.py:435: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
    Instructions for updating:
    Colocations handled automatically by placer.
    _________________________________________________________________
    Layer (type)                 Output Shape              Param #
    =================================================================
    input_1 (InputLayer)         (None, 1)                 0
    _________________________________________________________________
    dense (Dense)                (None, 20)                40
    _________________________________________________________________
    dense_1 (Dense)              (None, 20)                420
    _________________________________________________________________
    dense_2 (Dense)              (None, 1)                 21
    =================================================================
    Total params: 481
    Trainable params: 481
    Non-trainable params: 0
    _________________________________________________________________

```
def neg_log_likelihood(y_true, y_pred, sigma=1.0):
    dist = tfp.distributions.Normal(loc=y_pred, scale=sigma)
    return K.sum(-dist.log_prob(y_true))

model.compile(loss=neg_log_likelihood, optimizer=optimizers.Adam(lr=0.01), metrics=['mse'])
model.fit(X, y, batch_size=10, epochs=150, verbose=0);
```

```
from tqdm import tqdm_notebook

X_test = np.linspace(-0.8, 0.8, 1000).reshape(-1, 1)
y_pred_list = []

y_pred = model.predict(X_test)

y_mean = np.mean(y_pred, axis=1)
y_sigma = noise

plt.figure(figsize=(8,4))
plt.plot(X_test, y_mean, 'r-', label='Predictive mean');
plt.scatter(X, y, marker='+', label='Training data')
plt.fill_between(X_test.ravel(),
                 y_mean + 2 * y_sigma,
                 y_mean - 2 * y_sigma,
                 alpha=0.2, label='Uncertainty (Confidence Interval)')
plt.title('Prediction')
plt.legend();
```

#### Regression with an heteroscedastic noise model

This is equivalent to consider the following loss function:

$$
LogLoss = \sum_i \frac{(y_i - f(x_i))^2}{2 \sigma^2(x_i)}+\frac{1}{2} \log \sigma^2(x_i)
$$

```
# https://engineering.taboola.com/predicting-probability-distributions/

import numpy as np
import pandas as pd
import seaborn as sns
import tensorflow as tf
import matplotlib.pyplot as plt
from sklearn.utils import shuffle
def f(x):
    return x**2-6*x+9

def data_generator(x,sigma_0,samples):
    return np.random.normal(f(x),sigma_0*x,samples)

sigma_0 = 0.1
x_vals = np.arange(1,5.2,0.2)
x_arr = np.array([])
y_arr = np.array([])
samples = 50
for x in x_vals:
    x_arr = np.append(x_arr, np.full(samples,x))
    y_arr = np.append(y_arr, data_generator(x,sigma_0,samples))
x_arr, y_arr = shuffle(x_arr, y_arr)
x_test = np.arange(1.1,5.1,0.2)

fig, ax = plt.subplots(figsize=(10,10))
plt.grid(True)
plt.xlabel('x')
plt.ylabel('g(x)')
ax.scatter(x_arr,y_arr,label='sampled data')
ax.plot(x_vals,list(map(f,x_vals)),c='m',label='f(x)')
ax.legend(loc='upper center',fontsize='large',shadow=True)
plt.show()
```

```
epochs = 500
batch_size = 50
learning_rate = 0.0003
display_step = 50
batch_num = int(len(x_arr) / batch_size)

tf.reset_default_graph()
x = tf.placeholder(name='x',shape=(None,1),dtype=tf.float32)
y = tf.placeholder(name='y',shape=(None,1),dtype=tf.float32)

layer = x
for _ in range(3):
    layer = tf.layers.dense(inputs=layer, units=12, activation=tf.nn.tanh)

output = tf.layers.dense(inputs=layer, units=1)

# cot function
cost = tf.reduce_mean(tf.losses.mean_squared_error(labels=y,predictions=output))

optimizer = tf.train.AdamOptimizer(learning_rate).minimize(cost)
x_batches = np.array_split(x_arr, batch_num)
y_batches = np.array_split(y_arr, batch_num)

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    for epoch in range(epochs):
        avg_cost = 0.0
        x_batches, y_batches = shuffle(x_batches, y_batches)
        for i in range(batch_num):
            x_batch = np.expand_dims(x_batches[i],axis=1)
            y_batch = np.expand_dims(y_batches[i],axis=1)
            _, c = sess.run([optimizer,cost], feed_dict={x:x_batch, y:y_batch})
            avg_cost += c/batch_num
        if epoch % display_step == 0:
            print('Epoch {0} | cost = {1:.4f}'.format(epoch,avg_cost))
    y_pred = sess.run(output,feed_dict={x:np.expand_dims(x_test,axis=1)})
    print('Final cost: {0:.4f}'.format(avg_cost))

fig, ax = plt.subplots(figsize=(10,10))
plt.grid(True)
plt.xlabel('x')
plt.ylabel('y')
ax.scatter(x_arr,y_arr,c='b',label='sampled data')
ax.scatter(x_test,y_pred,c='r',label='predicted values')
ax.plot(x_vals,list(map(f,x_vals)),c='m',label='f(x)')
ax.legend(loc='upper center',fontsize='large',shadow=True)
plt.show()
```

```

#new cost function
def mdn_cost(mu, sigma, y):
    dist = tf.distributions.Normal(loc=mu, scale=sigma)
    return tf.reduce_mean(-dist.log_prob(y))

epochs = 500
batch_size = 50
learning_rate = 0.0003
display_step = 50
batch_num = int(len(x_arr) / batch_size)

tf.reset_default_graph()
x = tf.placeholder(name='x',shape=(None,1),dtype=tf.float32)
y = tf.placeholder(name='y',shape=(None,1),dtype=tf.float32)

layer = x
for _ in range(3):
    layer = tf.layers.dense(inputs=layer, units=12, activation=tf.nn.tanh)

# now we have two different outputs
mu = tf.layers.dense(inputs=layer, units=1)
sigma = tf.layers.dense(inputs=layer, units=1, activation=lambda x: tf.nn.elu(x) + 1)

cost = mdn_cost(mu, sigma, y)

optimizer = tf.train.AdamOptimizer(learning_rate).minimize(cost)
x_batches = np.array_split(x_arr, batch_num)
y_batches = np.array_split(y_arr, batch_num)
with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    for epoch in range(epochs):
        avg_cost = 0.0
        x_batches, y_batches = shuffle(x_batches, y_batches)
        for i in range(batch_num):
            x_batch = np.expand_dims(x_batches[i],axis=1)
            y_batch = np.expand_dims(y_batches[i],axis=1)
            _, c = sess.run([optimizer,cost], feed_dict={x:x_batch, y:y_batch})
            avg_cost += c/batch_num
        if epoch % display_step == 0:
            print('Epoch {0} | cost = {1:.4f}'.format(epoch,avg_cost))
    mu_pred, sigma_pred = sess.run([mu,sigma],feed_dict={x:np.expand_dims(x_test,axis=1)})
    print('Final cost: {0:.4f}'.format(avg_cost))

fig, ax = plt.subplots(figsize=(10,10))
plt.grid(True)
plt.xlabel('x')
plt.ylabel('y')
ax.errorbar(x_test,mu_pred,yerr=np.absolute(sigma_pred),c='r',ls='None',marker='.',ms=10,label='predicted distributions')
ax.scatter(x_arr,y_arr,c='b',alpha=0.05,label='sampled data')
ax.errorbar(x_vals,list(map(f,x_vals)),yerr=list(map(lambda x: sigma_0*x,x_vals)),c='b',lw=2,ls='None',marker='.',ms=10,label='true distributions')
ax.plot(x_vals,list(map(f,x_vals)),c='m',label='f(x)')
ax.legend(loc='upper center',fontsize='large',shadow=True)
plt.show()

```

## Epistemic and Total Uncertainty

##### Copyright 2019 The TensorFlow Authors.

Licensed under the Apache License, Version 2.0 (the "License");

```
#@title Licensed under the Apache License, Version 2.0 (the "License"); { display-mode: "form" }
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
```

# TFP Probabilistic Layers: Regression

    Run in Google Colab

    View source on GitHub

In this example we show how to fit regression models using TFP's "probabilistic layers."

### Dependencies & Prerequisites

```
#@title Install { display-mode: "form" }
TF_Installation = 'TF2 Nightly (GPU)' #@param ['TF2 Nightly (GPU)', 'TF2 Stable (GPU)', 'TF1 Nightly (GPU)', 'TF1 Stable (GPU)','System']

if TF_Installation == 'TF2 Nightly (GPU)':
  !pip install -q --upgrade tf-nightly-gpu-2.0-preview
  print('Installation of `tf-nightly-gpu-2.0-preview` complete.')
elif TF_Installation == 'TF2 Stable (GPU)':
  !pip install -q --upgrade tensorflow-gpu==2.0.0-alpha0
  print('Installation of `tensorflow-gpu==2.0.0-alpha0` complete.')
elif TF_Installation == 'TF1 Nightly (GPU)':
  !pip install -q --upgrade tf-nightly-gpu
  print('Installation of `tf-nightly-gpu` complete.')
elif TF_Installation == 'TF1 Stable (GPU)':
  !pip install -q --upgrade tensorflow-gpu
  print('Installation of `tensorflow-gpu` complete.')
elif TF_Installation == 'System':
  pass
else:
  raise ValueError('Selection Error: Please select a valid '
                   'installation option.')
```

         |████████████████████████████████| 349.2MB 52kB/s
         |████████████████████████████████| 3.1MB 38.7MB/s
         |████████████████████████████████| 61kB 26.3MB/s
         |████████████████████████████████| 430kB 59.4MB/s
      Building wheel for wrapt (setup.py) ... done
    ERROR: thinc 6.12.1 has requirement wrapt<1.11.0,>=1.10.0, but you'll have wrapt 1.11.1 which is incompatible.
    Installation of `tf-nightly-gpu-2.0-preview` complete.

```
#@title Install { display-mode: "form" }
TFP_Installation = "Nightly" #@param ["Nightly", "Stable", "System"]

if TFP_Installation == "Nightly":
  !pip install -q tfp-nightly
  print("Installation of `tfp-nightly` complete.")
elif TFP_Installation == "Stable":
  !pip install -q --upgrade tensorflow-probability
  print("Installation of `tensorflow-probability` complete.")
elif TFP_Installation == "System":
  pass
else:
  raise ValueError("Selection Error: Please select a valid "
                   "installation option.")
```

         |████████████████████████████████| 983kB 3.4MB/s
    Installation of `tfp-nightly` complete.

```
#@title Import { display-mode: "form" }

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

from pprint import pprint
import matplotlib.pyplot as plt
import numpy as np
import seaborn as sns

import tensorflow as tf
from tensorflow.python import tf2
if not tf2.enabled():
  import tensorflow.compat.v2 as tf
  tf.enable_v2_behavior()
  assert tf2.enabled()

import tensorflow_probability as tfp

sns.reset_defaults()
#sns.set_style('whitegrid')
#sns.set_context('talk')
sns.set_context(context='talk',font_scale=0.7)

%matplotlib inline

tfd = tfp.distributions
```

### Make things Fast!

Before we dive in, let's make sure we're using a GPU for this demo.

To do this, select "Runtime" -> "Change runtime type" -> "Hardware accelerator" -> "GPU".

The following snippet will verify that we have access to a GPU.

```
if tf.test.gpu_device_name() != '/device:GPU:0':
  print('WARNING: GPU device not found.')
else:
  print('SUCCESS: Found GPU: {}'.format(tf.test.gpu_device_name()))
```

    SUCCESS: Found GPU: /device:GPU:0

Note: if for some reason you cannot access a GPU, this colab will still work. (Training will just take longer.)

## Motivation

Wouldn't it be great if we could use TFP to specify a probabilistic model then simply minimize the negative log-likelihood, i.e.,

```
negloglik = lambda y, rv_y: -rv_y.log_prob(y)
```

Well not only is it possible, but this colab shows how! (In context of linear regression problems.)

```
#@title Synthesize dataset.
w0 = 0.125
b0 = 5.
x_range = [-20, 60]

def load_dataset(n=150, n_tst=150):
  np.random.seed(43)
  def s(x):
    g = (x - x_range[0]) / (x_range[1] - x_range[0])
    return 3 * (0.25 + g**2.)
  x = (x_range[1] - x_range[0]) * np.random.rand(n) + x_range[0]
  eps = np.random.randn(n) * s(x)
  y = (w0 * x * (1. + np.sin(x)) + b0) + eps
  x = x[..., np.newaxis]
  x_tst = np.linspace(*x_range, num=n_tst).astype(np.float32)
  x_tst = x_tst[..., np.newaxis]
  return y, x, x_tst

y, x, x_tst = load_dataset()
```

### Case 1: No Uncertainty

```
# Build model.
model = tf.keras.Sequential([
  tf.keras.layers.Dense(1),
  tfp.layers.DistributionLambda(lambda t: tfd.Normal(loc=t, scale=1)),
])

# Do inference.
model.compile(optimizer=tf.optimizers.Adam(learning_rate=0.01), loss=negloglik)
model.fit(x, y, epochs=1000, verbose=False);

# Profit.
[print(np.squeeze(w.numpy())) for w in model.weights];
yhat = model(x_tst)
assert isinstance(yhat, tfd.Distribution)
```

    0.13275796
    5.1289654

```
#@title Figure 1: No uncertainty.
w = np.squeeze(model.layers[-2].kernel.numpy())
b = np.squeeze(model.layers[-2].bias.numpy())

plt.figure(figsize=[6, 1.5])  # inches
#plt.figure(figsize=[8, 5])  # inches
plt.plot(x, y, 'b.', label='observed');
plt.plot(x_tst, yhat.mean(),'r', label='mean', linewidth=4);
plt.ylim(-0.,17);
plt.yticks(np.linspace(0, 15, 4)[1:]);
plt.xticks(np.linspace(*x_range, num=9));

ax=plt.gca();
ax.xaxis.set_ticks_position('bottom')
ax.yaxis.set_ticks_position('left')
ax.spines['left'].set_position(('data', 0))
ax.spines['top'].set_visible(False)
ax.spines['right'].set_visible(False)
#ax.spines['left'].set_smart_bounds(True)
#ax.spines['bottom'].set_smart_bounds(True)
plt.legend(loc='center left', fancybox=True, framealpha=0., bbox_to_anchor=(1.05, 0.5))

plt.savefig('/tmp/fig1.png', bbox_inches='tight', dpi=300)
```

### Case 2: Aleatoric Uncertainty

```
# Build model.
model = tf.keras.Sequential([
  tf.keras.layers.Dense(1 + 1),
  tfp.layers.DistributionLambda(
      lambda t: tfd.Normal(loc=t[..., :1],
                           scale=1e-3 + tf.math.softplus(0.05 * t[...,1:]))),
])

# Do inference.
model.compile(optimizer=tf.optimizers.Adam(learning_rate=0.01), loss=negloglik)
model.fit(x, y, epochs=1000, verbose=False);

# Profit.
[print(np.squeeze(w.numpy())) for w in model.weights];
yhat = model(x_tst)
assert isinstance(yhat, tfd.Distribution)
```

    [0.13226233 0.41329   ]
    [5.1153    1.2915019]

```
#@title Figure 2: Aleatoric Uncertainty
plt.figure(figsize=[6, 1.5])  # inches
plt.plot(x, y, 'b.', label='observed');

m = yhat.mean()
s = yhat.stddev()

plt.plot(x_tst, m, 'r', linewidth=4, label='mean');
plt.plot(x_tst, m + 2 * s, 'g', linewidth=2, label=r'mean + 2 stddev');
plt.plot(x_tst, m - 2 * s, 'g', linewidth=2, label=r'mean - 2 stddev');

plt.ylim(-0.,17);
plt.yticks(np.linspace(0, 15, 4)[1:]);
plt.xticks(np.linspace(*x_range, num=9));

ax=plt.gca();
ax.xaxis.set_ticks_position('bottom')
ax.yaxis.set_ticks_position('left')
ax.spines['left'].set_position(('data', 0))
ax.spines['top'].set_visible(False)
ax.spines['right'].set_visible(False)
#ax.spines['left'].set_smart_bounds(True)
#ax.spines['bottom'].set_smart_bounds(True)
plt.legend(loc='center left', fancybox=True, framealpha=0., bbox_to_anchor=(1.05, 0.5))

plt.savefig('/tmp/fig2.png', bbox_inches='tight', dpi=300)
```

### Case 3: Epistemic Uncertainty

```
# Specify the surrogate posterior over `keras.layers.Dense` `kernel` and `bias`.
def posterior_mean_field(kernel_size, bias_size=0, dtype=None):
  n = kernel_size + bias_size
  c = np.log(np.expm1(1.))
  return tf.keras.Sequential([
      tfp.layers.VariableLayer(2 * n, dtype=dtype),
      tfp.layers.DistributionLambda(lambda t: tfd.Independent(
          tfd.Normal(loc=t[..., :n],
                     scale=1e-5 + tf.nn.softplus(c + t[..., n:])),
          reinterpreted_batch_ndims=1)),
  ])
```

```
# Specify the surrogate posterior over `keras.layers.Dense` `kernel` and `bias`.
def prior_trainable(kernel_size, bias_size=0, dtype=None):
  n = kernel_size + bias_size
  return tf.keras.Sequential([
      tfp.layers.VariableLayer(n, dtype=dtype),
      tfp.layers.DistributionLambda(lambda t: tfd.Independent(
          tfd.Normal(loc=t, scale=1),
          reinterpreted_batch_ndims=1)),
  ])
```

```
# Build model.
model = tf.keras.Sequential([
  tfp.layers.DenseVariational(1, posterior_mean_field, prior_trainable),
  tfp.layers.DistributionLambda(lambda t: tfd.Normal(loc=t, scale=1)),
])

# Do inference.
model.compile(optimizer=tf.optimizers.Adam(learning_rate=0.01), loss=negloglik)
model.fit(x, y, epochs=1000, verbose=False);

# Profit.
[print(np.squeeze(w.numpy())) for w in model.weights];
yhat = model(x_tst)
assert isinstance(yhat, tfd.Distribution)
```

    [ 0.1374202  5.1056857 -3.7132006 -0.5256554]
    [0.1592448 5.12206  ]

```
#@title Figure 3: Epistemic Uncertainty
plt.figure(figsize=[6, 1.5])  # inches
plt.clf();
plt.plot(x, y, 'b.', label='observed');

yhats = [model(x_tst) for _ in range(100)]
avgm = np.zeros_like(x_tst[..., 0])
for i, yhat in enumerate(yhats):
  m = np.squeeze(yhat.mean())
  s = np.squeeze(yhat.stddev())
  if i < 25:
    plt.plot(x_tst, m, 'r', label='ensemble means' if i == 0 else None, linewidth=0.5)
  avgm += m
plt.plot(x_tst, avgm/len(yhats), 'r', label='overall mean', linewidth=4)

plt.ylim(-0.,17);
plt.yticks(np.linspace(0, 15, 4)[1:]);
plt.xticks(np.linspace(*x_range, num=9));

ax=plt.gca();
ax.xaxis.set_ticks_position('bottom')
ax.yaxis.set_ticks_position('left')
ax.spines['left'].set_position(('data', 0))
ax.spines['top'].set_visible(False)
ax.spines['right'].set_visible(False)
#ax.spines['left'].set_smart_bounds(True)
#ax.spines['bottom'].set_smart_bounds(True)
plt.legend(loc='center left', fancybox=True, framealpha=0., bbox_to_anchor=(1.05, 0.5))

plt.savefig('/tmp/fig3.png', bbox_inches='tight', dpi=300)
```

### Case 4: Aleatoric & Epistemic Uncertainty

```
# Build model.
model = tf.keras.Sequential([
  tfp.layers.DenseVariational(1 + 1, posterior_mean_field, prior_trainable),
  tfp.layers.DistributionLambda(
      lambda t: tfd.Normal(loc=t[..., :1],
                           scale=1e-3 + tf.math.softplus(0.01 * t[...,1:]))),
])

# Do inference.
model.compile(optimizer=tf.optimizers.Adam(learning_rate=0.01), loss=negloglik)
model.fit(x, y, epochs=1000, verbose=False);

# Profit.
[print(np.squeeze(w.numpy())) for w in model.weights];
yhat = model(x_tst)
assert isinstance(yhat, tfd.Distribution)
```

    [ 0.13091448  2.8183267   5.1185093   2.954348   -3.344546   -0.684905
     -0.5135757   0.05770317]
    [0.13267924 2.7312424  5.191225   2.9794762 ]

```
#@title Figure 4: Both Aleatoric & Epistemic Uncertainty
plt.figure(figsize=[6, 1.5])  # inches
plt.plot(x, y, 'b.', label='observed');

yhats = [model(x_tst) for _ in range(100)]
avgm = np.zeros_like(x_tst[..., 0])
for i, yhat in enumerate(yhats):
  m = np.squeeze(yhat.mean())
  s = np.squeeze(yhat.stddev())
  if i < 15:
    plt.plot(x_tst, m, 'r', label='ensemble means' if i == 0 else None, linewidth=1.)
    plt.plot(x_tst, m + 2 * s, 'g', linewidth=0.5, label='ensemble means + 2 ensemble stdev' if i == 0 else None);
    plt.plot(x_tst, m - 2 * s, 'g', linewidth=0.5, label='ensemble means - 2 ensemble stdev' if i == 0 else None);
  avgm += m
plt.plot(x_tst, avgm/len(yhats), 'r', label='overall mean', linewidth=4)

plt.ylim(-0.,17);
plt.yticks(np.linspace(0, 15, 4)[1:]);
plt.xticks(np.linspace(*x_range, num=9));

ax=plt.gca();
ax.xaxis.set_ticks_position('bottom')
ax.yaxis.set_ticks_position('left')
ax.spines['left'].set_position(('data', 0))
ax.spines['top'].set_visible(False)
ax.spines['right'].set_visible(False)
#ax.spines['left'].set_smart_bounds(True)
#ax.spines['bottom'].set_smart_bounds(True)
plt.legend(loc='center left', fancybox=True, framealpha=0., bbox_to_anchor=(1.05, 0.5))

plt.savefig('/tmp/fig4.png', bbox_inches='tight', dpi=300)
```


# Session 2 - Training a Network w/ Tensorflow

Assignment: Teach a Deep Neural Network to Paint

Parag K. Mital

Creative Applications of Deep Learning w/ Tensorflow

Kadenze Academy

#CADL

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

# Learning Goals

* Learn how to create a Neural Network
* Learn to use a neural network to paint an image
* Apply creative thinking to the inputs, outputs, and definition of a network

# Outline

- [Assignment Synopsis](#assignment-synopsis)
- [Part One - Fully Connected Network](#part-one---fully-connected-network)
    - [Instructions](#instructions)
    - [Code](#code)
    - [Variable Scopes](#variable-scopes)
- [Part Two - Image Painting Network](#part-two---image-painting-network)
    - [Instructions](#instructions-1)
    - [Preparing the Data](#preparing-the-data)
    - [Cost Function](#cost-function)
    - [Explore](#explore)
    - [A Note on Crossvalidation](#a-note-on-crossvalidation)
- [Part Three - Learning More than One Image](#part-three---learning-more-than-one-image)
    - [Instructions](#instructions-2)
    - [Code](#code-1)
- [Part Four - Open Exploration \(Extra Credit\)](#part-four---open-exploration-extra-credit)
- [Assignment Submission](#assignment-submission)

This next section will just make sure you have the right version of python and the libraries that we'll be using.  Don't change the code here but make sure you "run" it (use "shift+enter")!

```python
# First check the Python version
import sys
if sys.version_info < (3,4):
    print('You are running an older version of Python!\n\n' \
          'You should consider updating to Python 3.4.0 or ' \
          'higher as the libraries built for this course ' \
          'have only been tested in Python 3.4 and higher.\n')
    print('Try installing the Python 3.5 version of anaconda '
          'and then restart `jupyter notebook`:\n' \
          'https://www.continuum.io/downloads\n\n')

# Now get necessary libraries
try:
    import os
    import numpy as np
    import matplotlib.pyplot as plt
    from skimage.transform import resize
    from skimage import data
    from scipy.misc import imresize
except ImportError:
    print('You are missing some packages! ' \
          'We will try installing them before continuing!')
    !pip install "numpy>=1.11.0" "matplotlib>=1.5.1" "scikit-image>=0.11.3" "scikit-learn>=0.17" "scipy>=0.17.0"
    import os
    import numpy as np
    import matplotlib.pyplot as plt
    from skimage.transform import resize
    from skimage import data
    from scipy.misc import imresize
    print('Done!')

# Import Tensorflow
try:
    import tensorflow as tf
except ImportError:
    print("You do not have tensorflow installed!")
    print("Follow the instructions on the following link")
    print("to install tensorflow before continuing:")
    print("")
    print("https://github.com/pkmital/CADL#installation-preliminaries")

# This cell includes the provided libraries from the zip file
# and a library for displaying images from ipython, which
# we will use to display the gif
try:
    from libs import utils, gif
    import IPython.display as ipyd
except ImportError:
    print("Make sure you have started notebook in the same directory" +
          " as the provided zip file which includes the 'libs' folder" +
          " and the file 'utils.py' inside of it.  You will NOT be able"
          " to complete this assignment unless you restart jupyter"
          " notebook inside the directory created by extracting"
          " the zip file or cloning the github repo.")

# We'll tell matplotlib to inline any drawn figures like so:
%matplotlib inline
plt.style.use('ggplot')
```

```python
# Bit of formatting because I don't like the default inline code style:
from IPython.core.display import HTML
HTML("""<style> .rendered_html code {
    padding: 2px 4px;
    color: #c7254e;
    background-color: #f9f2f4;
    border-radius: 4px;
} </style>""")
```

# Assignment Synopsis

In this assignment, we're going to create our first neural network capable of taking any two continuous values as inputs.  Those two values will go through a series of multiplications, additions, and nonlinearities, coming out of the network as 3 outputs.  Remember from the last homework, we used convolution to filter an image so that the representations in the image were accentuated.  We're not going to be using convolution w/ Neural Networks until the next session, but we're effectively doing the same thing here: using multiplications to accentuate the representations in our data, in order to minimize whatever our cost function is.  To find out what those multiplications need to be, we're going to use Gradient Descent and Backpropagation, which will take our cost, and find the appropriate updates to all the parameters in our network to best optimize the cost.  In the next session, we'll explore much bigger networks and convolution.  This "toy" network is really to help us get up and running with neural networks, and aid our exploration of the different components that make up a neural network.  You will be expected to explore manipulations of the neural networks in this notebook as much as possible to help aid your understanding of how they effect the final result.

We're going to build our first neural network to understand what color "to paint" given a location in an image, or the row, col of the image.  So in goes a row/col, and out goes a R/G/B.  In the next lesson, we'll learn what this network is really doing is performing regression.  For now, we'll focus on the creative applications of such a network to help us get a better understanding of the different components that make up the neural network.  You'll be asked to explore many of the different components of a neural network, including changing the inputs/outputs (i.e. the dataset), the number of layers, their activation functions, the cost functions, learning rate, and batch size.  You'll also explore a modification to this same network which takes a 3rd input: an index for an image.  This will let us try to learn multiple images at once, though with limited success.

We'll now dive right into creating deep neural networks, and I'm going to show you the math along the way.  Don't worry if a lot of it doesn't make sense, and it really takes a bit of practice before it starts to come together.

# Part One - Fully Connected Network

## Instructions
Create the operations necessary for connecting an input to a network, defined by a `tf.Placeholder`, to a series of fully connected, or linear, layers, using the formula:

$$\textbf{H} = \phi(\textbf{X}\textbf{W} + \textbf{b})$$

where $\textbf{H}$ is an output layer representing the "hidden" activations of a network, $\phi$ represents some nonlinearity, $\textbf{X}$ represents an input to that layer, $\textbf{W}$ is that layer's weight matrix, and $\textbf{b}$ is that layer's bias.

If you're thinking, what is going on?  Where did all that math come from?  Don't be afraid of it.  Once you learn how to "speak" the symbolic representation of the equation, it starts to get easier.  And once we put it into practice with some code, it should start to feel like there is some association with what is written in the equation, and what we've written in code.  Practice trying to say the equation in a meaningful way: "The output of a hidden layer is equal to some input multiplied by another matrix, adding some bias, and applying a non-linearity".  Or perhaps: "The hidden layer is equal to a nonlinearity applied to an input multiplied by a matrix and adding some bias".  Explore your own interpretations of the equation, or ways of describing it, and it starts to become much, much easier to apply the equation.

The first thing that happens in this equation is the input matrix $\textbf{X}$ is multiplied by another matrix, $\textbf{W}$.  This is the most complicated part of the equation.  It's performing matrix multiplication, as we've seen from last session, and is effectively scaling and rotating our input.  The bias $\textbf{b}$ allows for a global shift in the resulting values.  Finally, the nonlinearity of $\phi$ allows the input space to be nonlinearly warped, allowing it to express a lot more interesting distributions of data.  Have a look below at some common nonlinearities.  If you're unfamiliar with looking at graphs like this, it is common to read the horizontal axis as X, as the input, and the vertical axis as Y, as the output.

```python
xs = np.linspace(-6, 6, 100)
plt.plot(xs, np.maximum(xs, 0), label='relu')
plt.plot(xs, 1 / (1 + np.exp(-xs)), label='sigmoid')
plt.plot(xs, np.tanh(xs), label='tanh')
plt.xlabel('Input')
plt.xlim([-6, 6])
plt.ylabel('Output')
plt.ylim([-1.5, 1.5])
plt.title('Common Activation Functions/Nonlinearities')
plt.legend(loc='lower right')
```

Remember, having series of linear followed by nonlinear operations is what makes neural networks expressive.  By stacking a lot of "linear" + "nonlinear" operations in a series, we can create a deep neural network!  Have a look at the output ranges of the above nonlinearity when considering which nonlinearity seems most appropriate.  For instance, the `relu` is always above 0, but does not saturate at any value above 0, meaning it can be anything above 0.  That's unlike the `sigmoid` which does saturate at both 0 and 1, meaning its values for a single output neuron will always be between 0 and 1.  Similarly, the `tanh` saturates at -1 and 1.

Choosing between these is often a matter of trial and error.  Though you can make some insights depending on your normalization scheme.  For instance, if your output is expected to be in the range of 0 to 1, you may not want to use a `tanh` function, which ranges from -1 to 1, but likely would want to use a `sigmoid`.  Keep the ranges of these activation functions in mind when designing your network, especially the final output layer of your network.

## Code

In this section, we're going to work out how to represent a fully connected neural network with code.  First, create a 2D `tf.placeholder` called $\textbf{X}$ with `None` for the batch size and 2 features.  Make its `dtype` `tf.float32`.  Recall that we use the dimension of `None` for the batch size dimension to say that this dimension can be any number.  Here is the docstring for the `tf.placeholder` function, have a look at what args it takes:

Help on function placeholder in module `tensorflow.python.ops.array_ops`:

```python
placeholder(dtype, shape=None, name=None)
```

    Inserts a placeholder for a tensor that will be always fed.

    **Important**: This tensor will produce an error if evaluated. Its value must
    be fed using the `feed_dict` optional argument to `Session.run()`,
    `Tensor.eval()`, or `Operation.run()`.

    For example:

```python
x = tf.placeholder(tf.float32, shape=(1024, 1024))
y = tf.matmul(x, x)

with tf.Session() as sess:
  print(sess.run(y))  # ERROR: will fail because x was not fed.

  rand_array = np.random.rand(1024, 1024)
  print(sess.run(y, feed_dict={x: rand_array}))  # Will succeed.
```

    Args:
      dtype: The type of elements in the tensor to be fed.
      shape: The shape of the tensor to be fed (optional). If the shape is not
        specified, you can feed a tensor of any shape.
      name: A name for the operation (optional).

    Returns:
      A `Tensor` that may be used as a handle for feeding a value, but not
      evaluated directly.

TODO! COMPLETE THIS SECTION!

```python
# Create a placeholder with None x 2 dimensions of dtype tf.float32, and name it "X":
X = ...
```

Now multiply the tensor using a new variable, $\textbf{W}$, which has 2 rows and 20 columns, so that when it is left mutiplied by $\textbf{X}$, the output of the multiplication is None x 20, giving you 20 output neurons.  Recall that the `tf.matmul` function takes two arguments, the left hand ($\textbf{X}$) and right hand side ($\textbf{W}$) of a matrix multiplication.

To create $\textbf{W}$, you will use `tf.get_variable` to create a matrix which is `2 x 20` in dimension.  Look up the docstrings of functions `tf.get_variable` and `tf.random_normal_initializer` to get familiar with these functions.  There are many options we will ignore for now.  Just be sure to set the `name`, `shape` (this is the one that has to be [2, 20]), `dtype` (i.e. tf.float32), and `initializer` (the `tf.random_normal_intializer` you should create) when creating your $\textbf{W}$ variable with `tf.get_variable(...)`.

For the random normal initializer, often the mean is set to 0, and the standard deviation is set based on the number of neurons.  But that really depends on the input and outputs of your network, how you've "normalized" your dataset, what your nonlinearity/activation function is, and what your expected range of inputs/outputs are.  Don't worry about the values for the initializer for now, as this part will take a bit more experimentation to understand better!

This part is to encourage you to learn how to look up the documentation on Tensorflow, ideally using `tf.get_variable?` in the notebook.  If you are really stuck, just scroll down a bit and I've shown you how to use it.

TODO! COMPLETE THIS SECTION!

```python
W = tf.get_variable(...
h = tf.matmul(...
```

And add to this result another new variable, $\textbf{b}$, which has [20] dimensions.  These values will be added to every output neuron after the multiplication above.  Instead of the `tf.random_normal_initializer` that you used for creating $\textbf{W}$, now use the `tf.constant_initializer`.  Often for bias, you'll set the constant bias initialization to 0 or 1.

TODO! COMPLETE THIS SECTION!

```python
b = tf.get_variable(...
h = tf.nn.bias_add(...
```

So far we have done:
$$\textbf{X}\textbf{W} + \textbf{b}$$

Finally, apply a nonlinear activation to this output, such as  `tf.nn.relu`, to complete the equation:

$$\textbf{H} = \phi(\textbf{X}\textbf{W} + \textbf{b})$$

TODO! COMPLETE THIS SECTION!

```python
h = ...
```

Now that we've done all of this work, let's stick it inside a function.  I've already done this for you and placed it inside the `utils` module under the function name `linear`.  We've already imported the `utils` module so we can call it like so, `utils.linear(...)`.  The docstring is copied below, and the code itself.  Note that this function is slightly different to the one in the lecture.  It does not require you to specify `n_input`, and the input `scope` is called `name`.  It also has a few more extras in there including automatically converting a 4-d input tensor to a 2-d tensor so that you can fully connect the layer with a matrix multiply (don't worry about what this means if it doesn't make sense!).

```python
utils.linear??
```

```python
def linear(x, n_output, name=None, activation=None, reuse=None):
    """Fully connected layer

    Parameters
    ----------
    x : tf.Tensor
        Input tensor to connect
    n_output : int
        Number of output neurons
    name : None, optional
        Scope to apply

    Returns
    -------
    op : tf.Tensor
        Output of fully connected layer.
    """
    if len(x.get_shape()) != 2:
        x = flatten(x, reuse=reuse)

    n_input = x.get_shape().as_list()[1]

    with tf.variable_scope(name or "fc", reuse=reuse):
        W = tf.get_variable(
            name='W',
            shape=[n_input, n_output],
            dtype=tf.float32,
            initializer=tf.contrib.layers.xavier_initializer())

        b = tf.get_variable(
            name='b',
            shape=[n_output],
            dtype=tf.float32,
            initializer=tf.constant_initializer(0.0))

        h = tf.nn.bias_add(
            name='h',
            value=tf.matmul(x, W),
            bias=b)

        if activation:
            h = activation(h)

        return h, W
```

## Variable Scopes

Note that since we are using `variable_scope` and explicitly telling the scope which name we would like, if there is *already* a variable created with the same name, then Tensorflow will raise an exception!  If this happens, you should consider one of three possible solutions:

1. If this happens while you are interactively editing a graph, you may need to reset the current graph:
```python
    tf.reset_default_graph()
```
You should really only have to use this if you are in an interactive console!  If you are creating Python scripts to run via command line, you should really be using solution 3 listed below, and be explicit with your graph contexts!
2. If this happens and you were not expecting any name conflicts, then perhaps you had a typo and created another layer with the same name!  That's a good reason to keep useful names for everything in your graph!
3. More likely, you should be using context managers when creating your graphs and running sessions.  This works like so:

    ```python
    g = tf.Graph()
    with tf.Session(graph=g) as sess:
        Y_pred, W = linear(X, 2, 3, activation=tf.nn.relu)
    ```

    or:

    ```python
    g = tf.Graph()
    with tf.Session(graph=g) as sess, g.as_default():
        Y_pred, W = linear(X, 2, 3, activation=tf.nn.relu)
    ```

You can now write the same process as the above steps by simply calling:

```python
h, W = utils.linear(
    x=X, n_output=20, name='linear', activation=tf.nn.relu)
```

# Part Two - Image Painting Network

## Instructions

Follow along the steps below, first setting up input and output data of the network, $\textbf{X}$ and $\textbf{Y}$. Then work through building the neural network which will try to compress the information in $\textbf{X}$ through a series of linear and non-linear functions so that whatever it is given as input, it minimized the error of its prediction, $\hat{\textbf{Y}}$, and the true output $\textbf{Y}$ through its training process.  You'll also create an animated GIF of the training which you'll need to submit for the homework!

Through this, we'll explore our first creative application: painting an image.  This network is just meant to demonstrate how easily networks can be scaled to more complicated tasks without much modification.  It is also meant to get you thinking about neural networks as building blocks that can be reconfigured, replaced, reorganized, and get you thinking about how the inputs and outputs can be anything you can imagine.

## Preparing the Data

We'll follow an example that Andrej Karpathy has done in his online demonstration of "image inpainting".  What we're going to do is teach the network to go from the location on an image frame to a particular color.  So given any position in an image, the network will need to learn what color to paint.  Let's first get an image that we'll try to teach a neural network to paint.

TODO! COMPLETE THIS SECTION!

```python
# First load an image
img = ...

# Be careful with the size of your image.
# Try a fairly small image to begin with,
# then come back here and try larger sizes.
img = imresize(img, (100, 100))
plt.figure(figsize=(5, 5))
plt.imshow(img)

# Make sure you save this image as "reference.png"
# and include it in your zipped submission file
# so we can tell what image you are trying to paint!
plt.imsave(fname='reference.png', arr=img)
```

In the lecture, I showed how to aggregate the pixel locations and their colors using a loop over every pixel position.  I put that code into a function `split_image` below.  Feel free to experiment with other features for `xs` or `ys`.

```python
def split_image(img):
    # We'll first collect all the positions in the image in our list, xs
    xs = []

    # And the corresponding colors for each of these positions
    ys = []

    # Now loop over the image
    for row_i in range(img.shape[0]):
        for col_i in range(img.shape[1]):
            # And store the inputs
            xs.append([row_i, col_i])
            # And outputs that the network needs to learn to predict
            ys.append(img[row_i, col_i])

    # we'll convert our lists to arrays
    xs = np.array(xs)
    ys = np.array(ys)
    return xs, ys
```

Let's use this function to create the inputs (xs) and outputs (ys) to our network as the pixel locations (xs) and their colors (ys):

```python
xs, ys = split_image(img)

# and print the shapes
xs.shape, ys.shape
```

Also remember, we should normalize our input values!

TODO! COMPLETE THIS SECTION!

```python
# Normalize the input (xs) using its mean and standard deviation
xs = ...

# Just to make sure you have normalized it correctly:
print(np.min(xs), np.max(xs))
assert(np.min(xs) > -3.0 and np.max(xs) < 3.0)
```

Similarly for the output:

```python
print(np.min(ys), np.max(ys))
```

We'll normalize the output using a simpler normalization method, since we know the values range from 0-255:

```python
ys = ys / 255.0
print(np.min(ys), np.max(ys))
```

Scaling the image values like this has the advantage that it is still interpretable as an image, unlike if we have negative values.

What we're going to do is use regression to predict the value of a pixel given its (row, col) position.  So the input to our network is `X = (row, col)` value.  And the output of the network is `Y = (r, g, b)`.

We can get our original image back by reshaping the colors back into the original image shape.  This works because the `ys` are still in order:

```python
plt.imshow(ys.reshape(img.shape))
```

But when we give inputs of (row, col) to our network, it won't know what order they are, because we will randomize them.  So it will have to *learn* what color value should be output for any given (row, col).

Create 2 placeholders of `dtype` `tf.float32`: one for the input of the network, a `None x 2` dimension placeholder called $\textbf{X}$, and another for the true output of the network, a `None x 3` dimension placeholder called $\textbf{Y}$.

TODO! COMPLETE THIS SECTION!

```python
# Let's reset the graph:
tf.reset_default_graph()

# Create a placeholder of None x 2 dimensions and dtype tf.float32
# This will be the input to the network which takes the row/col
X = tf.placeholder(...

# Create the placeholder, Y, with 3 output dimensions instead of 2.
# This will be the output of the network, the R, G, B values.
Y = tf.placeholder(...
```

Now create a deep neural network that takes your network input $\textbf{X}$ of 2 neurons, multiplies it by a linear and non-linear transformation which makes its shape [None, 20], meaning it will have 20 output neurons.  Then repeat the same process again to give you 20 neurons again, and then again and again until you've done 6 layers of 20 neurons.  Then finally one last layer which will output 3 neurons, your predicted output, which I've been denoting mathematically as $\hat{\textbf{Y}}$, for a total of 6 hidden layers, or 8 layers total including the input and output layers.  Mathematically, we'll be creating a deep neural network that looks just like the previous fully connected layer we've created, but with a few more connections. So recall the first layer's connection is:

\begin{align}
\textbf{H}_1=\phi(\textbf{X}\textbf{W}_1 + \textbf{b}_1) \\
\end{align}

So the next layer will take that output, and connect it up again:

\begin{align}
\textbf{H}_2=\phi(\textbf{H}_1\textbf{W}_2 + \textbf{b}_2) \\
\end{align}

And same for every other layer:

\begin{align}
\textbf{H}_3=\phi(\textbf{H}_2\textbf{W}_3 + \textbf{b}_3) \\
\textbf{H}_4=\phi(\textbf{H}_3\textbf{W}_4 + \textbf{b}_4) \\
\textbf{H}_5=\phi(\textbf{H}_4\textbf{W}_5 + \textbf{b}_5) \\
\textbf{H}_6=\phi(\textbf{H}_5\textbf{W}_6 + \textbf{b}_6) \\
\end{align}

Including the very last layer, which will be the prediction of the network:

\begin{align}
\hat{\textbf{Y}}=\phi(\textbf{H}_6\textbf{W}_7 + \textbf{b}_7)
\end{align}

Remember if you run into issues with variable scopes/names, that you cannot recreate a variable with the same name! Revisit the section on Variable Scopes if you get stuck with name issues.

TODO! COMPLETE THIS SECTION!

```python
# We'll create 6 hidden layers.  Let's create a variable
# to say how many neurons we want for each of the layers
# (try 20 to begin with, then explore other values)
n_neurons = ...

# Create the first linear + nonlinear layer which will
# take the 2 input neurons and fully connects it to 20 neurons.
# Use the `utils.linear` function to do this just like before,
# but also remember to give names for each layer, such as
# "1", "2", ... "5", or "layer1", "layer2", ... "layer6".
h1, W1 = ...

# Create another one:
h2, W2 = ...

# and four more (or replace all of this with a loop if you can!):
h3, W3 = ...
h4, W4 = ...
h5, W5 = ...
h6, W6 = ...

# Now, make one last layer to make sure your network has 3 outputs:
Y_pred, W7 = utils.linear(h6, 3, activation=None, name='pred')
```

```python
assert(X.get_shape().as_list() == [None, 2])
assert(Y_pred.get_shape().as_list() == [None, 3])
assert(Y.get_shape().as_list() == [None, 3])
```

## Cost Function

Now we're going to work on creating a `cost` function.  The cost should represent how much `error` there is in the network, and provide the optimizer this value to help it train the network's parameters using gradient descent and backpropagation.

Let's say our error is `E`, then the cost will be:

$$cost(\textbf{Y}, \hat{\textbf{Y}}) = \frac{1}{\text{B}} \displaystyle\sum\limits_{b=0}^{\text{B}} \textbf{E}_b
$$

where the error is measured as, e.g.:

$$\textbf{E} = \displaystyle\sum\limits_{c=0}^{\text{C}} (\textbf{Y}_{c} - \hat{\textbf{Y}}_{c})^2$$

Don't worry if this scares you.  This is mathematically expressing the same concept as: "the cost of an actual $\textbf{Y}$, and a predicted $\hat{\textbf{Y}}$ is equal to the mean across batches, of which there are $\text{B}$ total batches, of the sum of distances across $\text{C}$ color channels of every predicted output and true output".  Basically, we're trying to see on average, or at least within a single minibatches average, how wrong was our prediction?  We create a measure of error for every output feature by squaring the predicted output and the actual output it should have, i.e. the actual color value it should have output for a given input pixel position.  By squaring it, we penalize large distances, but not so much small distances.

Consider how the square function (i.e., $f(x) = x^2$) changes for a given error.  If our color values range between 0-255, then a typical amount of error would be between $0$ and $128^2$.  For example if my prediction was (120, 50, 167), and the color should have been (0, 100, 120), then the error for the Red channel is (120 - 0) or 120.  And the Green channel is (50 - 100) or -50, and for the Blue channel, (167 - 120) = 47.  When I square this result, I get: (120)^2, (-50)^2, and (47)^2.  I then add all of these and that is my error, $\textbf{E}$, for this one observation.  But I will have a few observations per minibatch.  So I add all the error in my batch together, then divide by the number of observations in the batch, essentially finding the mean error of my batch.

Let's try to see what the square in our measure of error is doing graphically.

```python
error = np.linspace(0.0, 128.0**2, 100)
loss = error**2.0
plt.plot(error, loss)
plt.xlabel('error')
plt.ylabel('loss')
```

This is known as the $l_2$ (pronounced el-two) loss.  It doesn't penalize small errors as much as it does large errors.  This is easier to see when we compare it with another common loss, the $l_1$ (el-one) loss.  It is linear in error, by taking the absolute value of the error.  We'll compare the $l_1$ loss with normalized values from $0$ to $1$.  So instead of having $0$ to $255$ for our RGB values, we'd have $0$ to $1$, simply by dividing our color values by $255.0$.

```python
error = np.linspace(0.0, 1.0, 100)
plt.plot(error, error**2, label='l_2 loss')
plt.plot(error, np.abs(error), label='l_1 loss')
plt.xlabel('error')
plt.ylabel('loss')
plt.legend(loc='lower right')
```

So unlike the $l_2$ loss, the $l_1$ loss is really quickly upset if there is *any* error at all: as soon as error moves away from $0.0$, to $0.1$, the $l_1$ loss is $0.1$.  But the $l_2$ loss is $0.1^2 = 0.01$.  Having a stronger penalty on smaller errors often leads to what the literature calls "sparse" solutions, since it favors activations that try to explain as much of the data as possible, rather than a lot of activations that do a sort of good job,  but when put together, do a great job of explaining the data.  Don't worry about what this means if you are more unfamiliar with Machine Learning.  There is a lot of literature surrounding each of these loss functions that we won't have time to get into, but look them up if they interest you.

During the lecture, we've seen how to create a cost function using Tensorflow.  To create a $l_2$ loss function, you can for instance use tensorflow's `tf.squared_difference` or for an $l_1$ loss function, `tf.abs`.  You'll need to refer to the `Y` and `Y_pred` variables only, and your resulting cost should be a single value.  Try creating the $l_1$ loss to begin with, and come back here after you have trained your network, to compare the performance with a $l_2$ loss.

The equation for computing cost I mentioned above is more succintly written as, for $l_2$ norm:

$$cost(\textbf{Y}, \hat{\textbf{Y}}) = \frac{1}{\text{B}} \displaystyle\sum\limits_{b=0}^{\text{B}} \displaystyle\sum\limits_{c=0}^{\text{C}} (\textbf{Y}_{c} - \hat{\textbf{Y}}_{c})^2$$

For $l_1$ norm, we'd have:

$$cost(\textbf{Y}, \hat{\textbf{Y}}) = \frac{1}{\text{B}} \displaystyle\sum\limits_{b=0}^{\text{B}} \displaystyle\sum\limits_{c=0}^{\text{C}} \text{abs}(\textbf{Y}_{c} - \hat{\textbf{Y}}_{c})$$

Remember, to understand this equation, try to say it out loud: the $cost$ given two variables, $\textbf{Y}$, the actual output we want the network to have, and $\hat{\textbf{Y}}$ the predicted output from the network, is equal to the mean across $\text{B}$ batches, of the sum of $\textbf{C}$ color channels distance between the actual and predicted outputs.  If you're still unsure, refer to the lecture where I've computed this, or scroll down a bit to where I've included the answer.

TODO! COMPLETE THIS SECTION!

```python
# first compute the error, the inner part of the summation.
# This should be the l1-norm or l2-norm of the distance
# between each color channel.
error = ...
assert(error.get_shape().as_list() == [None, 3])
```

TODO! COMPLETE THIS SECTION!

```python
# Now sum the error for each feature in Y.
# If Y is [Batch, Features], the sum should be [Batch]:
sum_error = ...
assert(sum_error.get_shape().as_list() == [None])
```

TODO! COMPLETE THIS SECTION!

```python
# Finally, compute the cost, as the mean error of the batch.
# This should be a single value.
cost = ...
assert(cost.get_shape().as_list() == [])
```

We now need an `optimizer` which will take our `cost` and a `learning_rate`, which says how far along the gradient to move.  This optimizer calculates all the gradients in our network with respect to the `cost` variable and updates all of the weights in our network using backpropagation.  We'll then create mini-batches of our training data and run the `optimizer` using a `session`.

TODO! COMPLETE THIS SECTION!

```python
# Refer to the help for the function
optimizer = tf.train....minimize(cost)

# Create parameters for the number of iterations to run for (< 100)
n_iterations = ...

# And how much data is in each minibatch (< 500)
batch_size = ...

# Then create a session
sess = tf.Session()
```

We'll now train our network!  The code below should do this for you if you've setup everything else properly.  Please read through this and make sure you understand each step!  Note that this can take a VERY LONG time depending on the size of your image (make it < 100 x 100 pixels), the number of neurons per layer (e.g. < 30), the number of layers (e.g. < 8), and number of iterations (< 1000).  Welcome to Deep Learning :)

```python
# Initialize all your variables and run the operation with your session
sess.run(tf.global_variables_initializer())

# Optimize over a few iterations, each time following the gradient
# a little at a time
imgs = []
costs = []
gif_step = n_iterations // 10
step_i = 0

for it_i in range(n_iterations):

    # Get a random sampling of the dataset
    idxs = np.random.permutation(range(len(xs)))

    # The number of batches we have to iterate over
    n_batches = len(idxs) // batch_size

    # Now iterate over our stochastic minibatches:
    for batch_i in range(n_batches):

        # Get just minibatch amount of data
        idxs_i = idxs[batch_i * batch_size: (batch_i + 1) * batch_size]

        # And optimize, also returning the cost so we can monitor
        # how our optimization is doing.
        training_cost = sess.run(
            [cost, optimizer],
            feed_dict={X: xs[idxs_i], Y: ys[idxs_i]})[0]

    # Also, every 20 iterations, we'll draw the prediction of our
    # input xs, which should try to recreate our image!
    if (it_i + 1) % gif_step == 0:
        costs.append(training_cost / n_batches)
        ys_pred = Y_pred.eval(feed_dict={X: xs}, session=sess)
        img = np.clip(ys_pred.reshape(img.shape), 0, 1)
        imgs.append(img)
        # Plot the cost over time
        fig, ax = plt.subplots(1, 2)
        ax[0].plot(costs)
        ax[0].set_xlabel('Iteration')
        ax[0].set_ylabel('Cost')
        ax[1].imshow(img)
        fig.suptitle('Iteration {}'.format(it_i))
        plt.show()
```

```python
# Save the images as a GIF
_ = gif.build_gif(imgs, saveto='single.gif', show_gif=False)
```

Let's now display the GIF we've just created:

```python
ipyd.Image(url='single.gif?{}'.format(np.random.rand()),
           height=500, width=500)
```

## Explore

Go back over the previous cells and exploring changing different parameters of the network.  I would suggest first trying to change the `learning_rate` parameter to different values and see how the cost curve changes.  What do you notice?  Try exponents of $10$, e.g. $10^1$, $10^2$, $10^3$... and so on.  Also try changing the `batch_size`: $50, 100, 200, 500, ...$ How does it effect how the cost changes over time?

Be sure to explore other manipulations of the network, such as changing the loss function to $l_2$ or $l_1$.  How does it change the resulting learning?  Also try changing the activation functions, the number of layers/neurons, different optimizers, and anything else that you may think of, and try to get a basic understanding on this toy problem of how it effects the network's training.  Also try comparing creating a fairly shallow/wide net (e.g. 1-2 layers with many neurons, e.g. > 100), versus a deep/narrow net (e.g. 6-20 layers with fewer neurons, e.g. < 20).  What do you notice?

## A Note on Crossvalidation

The cost curve plotted above is only showing the cost for our "training" dataset.  Ideally, we should split our dataset into what are called "train", "validation", and "test" sets.  This is done by taking random subsets of the entire dataset.  For instance, we partition our dataset by saying we'll only use 80% of it for training, 10% for validation, and the last 10% for testing.  Then when training as above, you would only use the 80% of the data you had partitioned, and then monitor accuracy on both the data you have used to train, but also that new 10% of unseen validation data.  This gives you a sense of how "general" your network is.  If it is performing just as well on that 10% of data, then you know it is doing a good job.  Finally, once you are done training, you would test one last time on your "test" dataset.  Ideally, you'd do this a number of times, so that every part of the dataset had a chance to be the test set.  This would also give you a measure of the variance of the accuracy on the final test.  If it changes a lot, you know something is wrong.  If it remains fairly stable, then you know that it is a good representation of the model's accuracy on unseen data.

We didn't get a chance to cover this in class, as it is less useful for exploring creative applications, though it is very useful to know and to use in practice, as it avoids overfitting/overgeneralizing your network to all of the data.  Feel free to explore how to do this on the application above!

# Part Three - Learning More than One Image

## Instructions

We're now going to make use of our Dataset from Session 1 and apply what we've just learned to try and paint every single image in our dataset.  How would you guess is the best way to approach this?  We could for instance feed in every possible image by having multiple row, col -> r, g, b values.  So for any given row, col, we'd have 100 possible r, g, b values.  This likely won't work very well as there are many possible values a pixel could take, not just one.  What if we also tell the network *which* image's row and column we wanted painted?  We're going to try and see how that does.

You can execute all of the cells below unchanged to see how this works with the first 100 images of the celeb dataset.  But you should replace the images with your own dataset, and vary the parameters of the network to get the best results!

I've placed the same code for running the previous algorithm into two functions, `build_model` and `train`.  You can directly call the function `train` with a 4-d image shaped as N x H x W x C, and it will collect all of the points of every image and try to predict the output colors of those pixels, just like before.  The only difference now is that you are able to try this with a few images at a time.  There are a few ways we could have tried to handle multiple images.  The way I've shown in the `train` function is to include an additional input neuron for *which* image it is.  So as well as receiving the row and column, the network will also receive as input which image it is as a number.  This should help the network to better distinguish the patterns it uses, as it has knowledge that helps it separates its process based on which image is fed as input.

```python
def build_model(xs, ys, n_neurons, n_layers, activation_fn,
                final_activation_fn, cost_type):

    xs = np.asarray(xs)
    ys = np.asarray(ys)

    if xs.ndim != 2:
        raise ValueError(
            'xs should be a n_observates x n_features, ' +
            'or a 2-dimensional array.')
    if ys.ndim != 2:
        raise ValueError(
            'ys should be a n_observates x n_features, ' +
            'or a 2-dimensional array.')

    n_xs = xs.shape[1]
    n_ys = ys.shape[1]

    X = tf.placeholder(name='X', shape=[None, n_xs],
                       dtype=tf.float32)
    Y = tf.placeholder(name='Y', shape=[None, n_ys],
                       dtype=tf.float32)

    current_input = X
    for layer_i in range(n_layers):
        current_input = utils.linear(
            current_input, n_neurons,
            activation=activation_fn,
            name='layer{}'.format(layer_i))[0]

    Y_pred = utils.linear(
        current_input, n_ys,
        activation=final_activation_fn,
        name='pred')[0]

    if cost_type == 'l1_norm':
        cost = tf.reduce_mean(tf.reduce_sum(
                tf.abs(Y - Y_pred), 1))
    elif cost_type == 'l2_norm':
        cost = tf.reduce_mean(tf.reduce_sum(
                tf.squared_difference(Y, Y_pred), 1))
    else:
        raise ValueError(
            'Unknown cost_type: {}.  '.format(
            cost_type) + 'Use only "l1_norm" or "l2_norm"')

    return {'X': X, 'Y': Y, 'Y_pred': Y_pred, 'cost': cost}
```

```python
def train(imgs,
          learning_rate=0.0001,
          batch_size=200,
          n_iterations=10,
          gif_step=2,
          n_neurons=30,
          n_layers=10,
          activation_fn=tf.nn.relu,
          final_activation_fn=tf.nn.tanh,
          cost_type='l2_norm'):

    N, H, W, C = imgs.shape
    all_xs, all_ys = [], []
    for img_i, img in enumerate(imgs):
        xs, ys = split_image(img)
        all_xs.append(np.c_[xs, np.repeat(img_i, [xs.shape[0]])])
        all_ys.append(ys)
    xs = np.array(all_xs).reshape(-1, 3)
    xs = (xs - np.mean(xs, 0)) / np.std(xs, 0)
    ys = np.array(all_ys).reshape(-1, 3)
    ys = ys / 127.5 - 1

    g = tf.Graph()
    with tf.Session(graph=g) as sess:
        model = build_model(xs, ys, n_neurons, n_layers,
                            activation_fn, final_activation_fn,
                            cost_type)
        optimizer = tf.train.AdamOptimizer(
            learning_rate=learning_rate).minimize(model['cost'])
        sess.run(tf.global_variables_initializer())
        gifs = []
        costs = []
        step_i = 0
        for it_i in range(n_iterations):
            # Get a random sampling of the dataset
            idxs = np.random.permutation(range(len(xs)))

            # The number of batches we have to iterate over
            n_batches = len(idxs) // batch_size
            training_cost = 0

            # Now iterate over our stochastic minibatches:
            for batch_i in range(n_batches):

                # Get just minibatch amount of data
                idxs_i = idxs[batch_i * batch_size:
                              (batch_i + 1) * batch_size]

                # And optimize, also returning the cost so we can monitor
                # how our optimization is doing.
                cost = sess.run(
                    [model['cost'], optimizer],
                    feed_dict={model['X']: xs[idxs_i],
                               model['Y']: ys[idxs_i]})[0]
                training_cost += cost

            print('iteration {}/{}: cost {}'.format(
                    it_i + 1, n_iterations, training_cost / n_batches))

            # Also, every 20 iterations, we'll draw the prediction of our
            # input xs, which should try to recreate our image!
            if (it_i + 1) % gif_step == 0:
                costs.append(training_cost / n_batches)
                ys_pred = model['Y_pred'].eval(
                    feed_dict={model['X']: xs}, session=sess)
                img = ys_pred.reshape(imgs.shape)
                gifs.append(img)
        return gifs
```

## Code

Below, I've shown code for loading the first 100 celeb files.  Run through the next few cells to see how this works with the celeb dataset, and then come back here and replace the `imgs` variable with your own set of images.  For instance, you can try your entire sorted dataset from Session 1 as an N x H x W x C array.  Explore!

TODO! COMPLETE THIS SECTION!

```python
celeb_imgs = utils.get_celeb_imgs()
plt.figure(figsize=(10, 10))
plt.imshow(utils.montage(celeb_imgs).astype(np.uint8))
# It doesn't have to be 100 images, explore!
imgs = np.array(celeb_imgs).copy()
```

Explore changing the parameters of the `train` function and your own dataset of images.  Note, you do not have to use the dataset from the last assignment!  Explore different numbers of images, whatever you prefer.

TODO! COMPLETE THIS SECTION!

```python
# Change the parameters of the train function and
# explore changing the dataset
gifs = train(imgs=imgs)
```

Now we'll create a gif out of the training process.  Be sure to call this 'multiple.gif' for your homework submission:

```python
montage_gifs = [np.clip(utils.montage(
            (m * 127.5) + 127.5), 0, 255).astype(np.uint8)
                for m in gifs]
_ = gif.build_gif(montage_gifs, saveto='multiple.gif')
```

And show it in the notebook

```python
ipyd.Image(url='multiple.gif?{}'.format(np.random.rand()),
           height=500, width=500)
```

What we're seeing is the training process over time.  We feed in our `xs`, which consist of the pixel values of each of our 100 images, it goes through the neural network, and out come predicted color values for every possible input value.  We visualize it above as a gif by seeing how at each iteration the network has predicted the entire space of the inputs.  We can visualize just the last iteration as a "latent" space, going from the first image (the top left image in the montage), to the last image, (the bottom right image).

```python
final = gifs[-1]
final_gif = [np.clip(((m * 127.5) + 127.5), 0, 255).astype(np.uint8) for m in final]
gif.build_gif(final_gif, saveto='final.gif')
```

```python
ipyd.Image(url='final.gif?{}'.format(np.random.rand()),
           height=200, width=200)
```

# Part Four - Open Exploration (Extra Credit)

I now what you to explore what other possible manipulations of the network and/or dataset you could imagine.  Perhaps a process that does the reverse, tries to guess where a given color should be painted?  What if it was only taught a certain palette, and had to reason about other colors, how it would interpret those colors?  Or what if you fed it pixel locations that weren't part of the training set, or outside the frame of what it was trained on?  Or what happens with different activation functions, number of layers, increasing number of neurons or lesser number of neurons?  I leave any of these as an open exploration for you.

Try exploring this process with your own ideas, materials, and networks, and submit something you've created as a gif!   To aid exploration, be sure to scale the image down quite a bit or it will require a much larger machine, and much more time to train.  Then whenever you think you may be happy with the process you've created, try scaling up the resolution and leave the training to happen over a few hours/overnight to produce something truly stunning!

Make sure to name the result of your gif: "explore.gif", and be sure to include it in your zip file.

TODO! COMPLETE THIS SECTION!

```python
# Train a network to produce something, storing every few
# iterations in the variable gifs, then export the training
# over time as a gif.
...

gif.build_gif(montage_gifs, saveto='explore.gif')
```

```python
ipyd.Image(url='explore.gif?{}'.format(np.random.rand()),
           height=500, width=500)
```

# Assignment Submission

After you've completed the notebook, create a zip file of the current directory using the code below.  This code will make sure you have included this completed ipython notebook and the following files named exactly as:

    session-2/
      session-2.ipynb
      single.gif
      multiple.gif
      final.gif
      explore.gif*
      libs/
        utils.py

    * = optional/extra-credit

You'll then submit this zip file for your second assignment on Kadenze for "Assignment 2: Teach a Deep Neural Network to Paint"!  If you have any questions, remember to reach out on the forums and connect with your peers or with me.

To get assessed, you'll need to be a premium student!  This will allow you to build an online portfolio of all of your work and receive grades.  If you aren't already enrolled as a student, register now at http://www.kadenze.com/ and join the [#CADL](https://twitter.com/hashtag/CADL) community to see what your peers are doing! https://www.kadenze.com/courses/creative-applications-of-deep-learning-with-tensorflow/info

Also, if you share any of the GIFs on Facebook/Twitter/Instagram/etc..., be sure to use the #CADL hashtag so that other students can find your work!

```python
utils.build_submission('session-2.zip',
                       ('reference.png',
                        'single.gif',
                        'multiple.gif',
                        'final.gif',
                        'session-2.ipynb'),
                       ('explore.gif'))
```