DeepMind x UCL | Deep Learning Lectures | 10/12 |  Unsupervised Representation Learning

hi my name is Irina and I'm a research
scientist at deep mind we're together
with mihaela I work in the frontiers
team
staying true to our team name today
mihaela and I will talk about the
frontiers in deep learning and in
particular about the role of
unsupervised representation learning in
expanding these frontiers if you listen
to any of the previous lectures in the
series you would have learned about some
of the amazing things more than deep
learning algorithms can achieve today
we'll take a slightly more critical
approach and think about what such
algorithms so struggle with and how
representation learning fits into the
big picture as a potential solution to
some of these outstanding problems we
will start with a broad overview of
unsupervised learning including its
historical role in the broader field of
machine learning we will then move on to
thinking about why unsupervised learning
may be important before taking a
multidisciplinary approach to reasoning
about what may constitute a good
representation and how we might evaluate
whether we are on the right path note
that throughout the lecture we will use
the term unsupervised learning to refer
to self supervised learning - in parts
five and six
mihaela will take over and talk about
some of the major existing techniques
and applications for unsupervised
representation learning and future
research directions in the field while
unfortunately it is impossible to cover
all the amazing work relevant to this
area of research we try to include some
of the key references in the yellow want
to learn more boxes which will appear
throughout our talk if you are
interested in learning more about any of
the topics covered in this lecture we
encourage you to follow the citations
surrounding convention papers let's
start with looking at what is
unsupervised learning unsupervised
learning is one of the three major
branches in machine learning alongside
supervised and reinforcement learning in
supervised learning every input example
is accompanied by its corresponding
label for example if we want to learn
how to classify robots our training data
may consist of images of different
robots and their names essentially the
goal of supervised learning is to learn
a mapping from the given inputs to the
given outputs that hopefully generalizes
to other examples from the same
distribution the goal of reinforcement
learning or RL is to discover which
actions to take at any state in order to
maximize the expected discounted future
reward for example if the task for the
robot is to reach the star it should
know that in its current state it is
optimal to go in the right or down but
not up or left note however that the
robot will not get feedback on every
action it takes whether the action was
good or not instead it will only get
feedback once it solves the task in
terms of single scalar reward
so unlike supervised learning where
reached each teaching signal is provided
for every observation in reinforcement
learning teacher signals are much poorer
and poorer note that in the rest of the
talk and many sometimes abuse the term
supervised learning to refer to both
supervised and reinforcement learning
since they both require some external
supervision to operate in unsupervised
learning we are in the extreme situation
of having no teacher signal whatsoever
all we have access to is the input data
and we have to figure out what to do
with it
which raises two questions do we need
unsupervised learning and if we do how
do we evaluate whether the algorithm we
have come up with is good let's start
with the first question if the only
thing we have access to is a bunch of
data observations what can we do with it
one thing we can do is try to find some
structure in it are some observations
more similar to each other than others
can we cluster similar data points
together clustering is important because
because it allows us to solve subsequent
classification problems with much less
data indeed it's not to learn something
about one of the robots in each cluster
to generalize this knowledge
automatically to all the other examples
another related thing we can do is to
find a small set of axes that explain
the majority of the variation in our
data for example if all the major
differences between our robots can be
explained in terms of their height and
we'll number it may be better to
represent each each robot as a point in
this two-dimensional space rather than
as a significantly high dimensional
image this may make the data more
interpretable to a human it will also
make many classification tasks on top of
such a representation easier to solve
the other question to consider when
thinking about unsupervised learning is
how to evaluate rather than an algorithm
we have come up with is doing something
good considering that we don't have
ground truth to competitor this happens
to be a really challenging question if
we wanted to find classes in the data
how do we know which classes are good
for example we could cluster by leg type
or arm number or height each choice
results in a different representation of
the data all of which are equally valid
how do we know which of the choices are
good and which are bad similar
considerations arise when we try to
reduce the dimensionality of the data do
we prioritize orthogonality of the
discovered bases do we care about
finding the independent sources that
explain our signal or do we look for
something else perhaps human interpreted
bill
so far we have seen a natural way to use
unsupervised learning is a useful
pre-processing step to find a
representation that will improve the
general interpretability of the data all
the ability to solve subsequent
supervised tasks however how necessary
is that step given the challenges
surrounding the question of how to
evaluate unsupervised algorithms and the
representations they discover one may
wonder should we bother at all why not
just develop better supervised
algorithms instead of explicitly
thinking of unsupervised representation
learning to start answering this
question let's take a bird's eye view on
the history of machine learning and the
role of representation learning in it
it's always hard to know why to start
historical timelines like this but the
useful point to consider is 1949 which
as far as I know was the first written
use of the term machine learning in a
scientific publication when Arthur
Samuel introduced a checker playing AI
since then and up to around 2006
representations played a major role in
machine learning from handcrafted
feature engineering where data
scientists would manually create input
features to their models to the hugely
popular and successful use of kernel
methods finding good representations of
the given data was at the core of
machine learning success 2006 can be
seen as a major point for deep
unsupervised representation learning
when Hinton answer CUDA Nev introduced
the use of three training restricted
Boltzmann machines for initializing deep
neural networks for improved convergence
on classification tasks things turned
sour however once alex nod was shown to
be able to win the imagenet challenge
for image classification without the
need for any unsupervised retraining
suddenly and supervisor presentation
learning sink obsolete with supervised
deep neural networks being able to the
discover representations for solving
tasks implicitly through end-to-end
gradient based optimization from 2012
onwards it seemed like as a field we had
cracked the recipe for machine learning
success more data deeper models by the
hardware this generated a huge rush of
excitement which was well justified to
this recipe did produce amazing result
from game playing deep reinforcement
learning agents that were able to beat
humans world champions to incredible
improvements in machine translation text
to speech and self-driving car
technology to name a few
this made some people question is
machine learning solved unfortunately
this wasn't quite the case as what was
pointed out by more and more papers
discussing the shortcomings of the
state-of-the-art deep supervised and
reinforcement learning algorithms date
efficiency for once was shown to be
disproportionately poor for the existing
algorithms compared to human learning
for example Brendon Lake showed how one
of the best are deep reinforcement
learning algorithms for playing Atari
dqn took orders of magnitude more time
to learn how to play frostbite compared
to a human player this may not be a
problem for tasks where a lot of data is
easily available but in many real-life
applications like robotics data
collection is a problem which makes date
efficiency an important shortcoming to
address another often mentioned problem
is to do with the robustness of deep
algorithms for example an image
classifier that can perform as well as
humans on differentiating between
thousands of objects can easily succumb
to something called an adversarial
attack the image shown above is
correctly classified as that of a panda
with 57 percent confidence however when
a tiny bit of modest is added to it
producing another image of a panda that
to any human would be indistinguishable
from the first one the algorithm totally
fails and misclassified misclassifies
the image as a Gibbon with 99% certainty
this can lead to potentially
catastrophic implications in real-life
scenarios for example if an artificial
visual system like this is installed in
a self-driving car and its ability to
read road signs is disturbed by enter
Sara Lee place patches as was recently
discussed in the nature news article
linked another major problem with many
current algorithms is generalization for
example as a human once we learn how to
play a game our performance should not
be affected by irrelevant changes to the
game visuals but changes in the
background color however many current
algorithms do quite poorly in such
scenarios as was demonstrated by the
open AI team here we see that two
state-of-the-art deep RL algorithms
generalized quite poorly as shown by the
low red curve plotting performance on
unseen variations of the game compared
to the blue curve showing performance on
the training levels note that the gap
persists even when the algorithms are
trained on tens of thousands of
variations of the rather simple coin
ranking demonstrating inability to grasp
the core ideas of the game that would
generalize across irrelevant variations
related to this transfer is the ability
to reuse previously acquired knowledge
in new situations for example knowing
how to play the Atari game breakout
where you move the paddle to keep the
ball in the air should generalize to new
versions of the game where an extra wall
is introduced or the panel is slightly
offset or when the colored blocks are
removed and the game is just about
juggling the balls in the air current
state-of-the-art models are not able to
do that as was demonstrated by the team
from the carriers which reported
significant drops in the scores
achieved on the variations of the game
for a state of the art RL model finally
all our existing algorithms tend to lack
what we might call common sense as
demonstrated by these images image
caption examples from Brandon Lakes
paper why this algorithm clearly has a
good language model and understands
individual objects in the image it
appears to struggle with the notions of
causality intuitive physics and abstract
concepts indeed the shift and focus from
designing algorithms that can solve
single tasks through end-to-end
supervision to building algorithms that
are more general and can achieve good
performance across multiple tasks in the
data efficient manner is trending with a
number of top AI industrial research
labs announcing challenges to push the
state-of-the-art in semi-supervised
learning like google's visual task
adaptation benchmark generalization with
open a is coin run and transfer with the
vines TM lab challenge to sunrise
we'll find ourselves in a situation
where we have more or less worked out
the formula for how to solve single
tasks with entrant deep learning given
up data and compute however if we want
to move to the next generation of aoa
which is more debt efficient efficient
robust and general generalizable across
multiple tasks we need a new paradigm
what might it be the three leading minds
in AI and recent term Turing Award
winners young a cool job Hinton and
yoshua bengio seem to think that the
answer is unsupervised or presentation
learning quoting them from the recent
Triple A conference that took place in
January this year
I always knew a supervised learning was
the right thing to do basically is the
idea of learning to represent the world
before learning a task and that is what
babies do
and so if we can build models of the
world where we have the right
abstractions where we can pin down those
changes to just one or a few variables
then we will be able to adapt to those
challenges because we don't need as much
data as much observation in order to
figure out what has changed if we were
to follow that valise and improve
unsupervised representation learning it
is important to consider in advance what
kinds of representations we might want
to learn does the representation has to
be flat or hierarchical should it be
learned once and frozen or continuously
adapted if we're going to learn a
representation that would make solving
as many subsequent tasks as possible
easy what would it look like these are
the kinds of questions we are going to
ask in the next section so what makes a
good representation this is a
surprisingly ill-defined and challenging
question when faced with a difficult
problem like this one possible strategy
is to look two related disciplines for
inspiration when it comes to thinking
about good representations which would
support intelligence behavior on many
tasks one natural place to turn to is
neuroscience since it has been
interested in the nature of
representations in biological
intelligence since its inception let's
start with some definitions in
neuroscience and cognitive science a
representation is thought of as a formal
system for making explicit certain
entities or types of information
together with a specification of how the
system does this let's unpack this with
an example on the screen you see three
ways of writing the number 37 note that
while the information content is the
same in all three cases their
representation or form is different this
implies that how information is
represented is orthogonal to what this
information is so
representation is a useful abstraction
and the choice of a presentational form
is important because it makes different
types of computations more efficient for
example the Arabic numeral 37 makes
explicit it's the composition in two
powers of 10 while the binary form makes
explicit it's the composition in two
powers of two finally in order to truly
understand a representational form we
have to think about the full shape of
the manifold that the data should lie on
rather than thinking of a individual
data point in isolation let's put this
in practice what happens to the
representational form in the brain as
visual information moves from the retina
to the visual cortex let's see what
happens when an image of a car gets
projected on the retina at this point
this image becomes a point in the high
dimensional space of firing rates of the
early visual neurons all possible
identity preserving transformations of
the car for example its rotations and
changes in size will then form a
manifold of points in this high
dimensional space of neural responses
the untangling hypothesis suggests that
the car manifold as well as other object
manifolds are all tangled up at the
early stages of visual processes this
tangling makes tasks like object scatter
categorization hard since the manifold
for different objects are difficult to
separate from each other with a simple
decision boundary the role of the
ventral visual stream then becomes to
change the representational form in a
way that untangles object manifold
making the subsequent task decision
boundaries simpler note that all the
information about the objects that might
be necessary to solve future tasks is
already present at the retina the role
of visual processing is just to reformat
that information into a better
representation of form
to representations have any other
properties apart from being untangled
it appears that approaching this
question from the reinforcement learning
perspective and thinking about
representations as states forming a
Markov decision process or NDP use for
solid tasks can shed light on what
properties which representations should
have let's imagine a concrete scenario
let's say you find yourself at a busy
intersection and your task is to reach
home safely what would be a good way to
represent your current state to solve
this simple task
it appears that depending on your
representational choices your journey
home may be guaranteed to be safe like
in the MVP visualized in the lowest
chaotic or you might always be in danger
of being hit by a car as shown in the
top schematic the only difference
between the two MVPs is that in the
lower one our representation of the
current state contains information about
the presence of cars the splits the
single state of being on the sidewalk
into two states being there with no cars
or with a car approaching the simple
choice of what information to include in
a representation can they can therefore
significantly affect the difficulty of a
task to solve so I think extra
information about the presence of a car
helped does it mean that our
representation should have also included
all the types of information that are
readily available on our retina like
what time of day it is or what color the
approaching car is the answer is no we
want to exclude all information
irrelevant to solve it a task to be able
to generalize our learnings across
things like time of day or which car is
approaching this implies that our
representation should support easy
attentional attenuation to remove
unimportant details which in turn would
also allow for flexible clustering
together of perceptually different
experiences for better general
sation note that sometimes we may not
only want to cross the perceptual
different things together but we may
also want to pull perceptually similar
things apart for example imagine that
instead of trying to cross the strait
the task is now to hail a taxi
even though perceptually who would be in
the same state on the same sidewalk as
before we would want to have a different
representation for the task of hailing a
cab compared to their going home task
for once if we're hailing a cab who
would want to keep around the
information about the color of the car
to know when a yellow cab is approaching
exactly the information we wanted to
discard for improved generalization in
the previous scenario this implies that
our representation should also support
input from latent sources of information
like which task were solving in order to
be adjusted through attentional
processes accordingly another important
idea to consider is that of
compositionality the key ingredient that
makes human language so special in
linguistics compositionality refers to
the ability to infer the meaning of a
complex expression from the meanings of
its constituent parts and the rules used
to combine them for example the sentence
I saw the man with the binoculars can be
interpreted in two different ways
depending on which of the two syntactic
parsing trees is inferred in the first
instance the man we're seeing has the
binoculars while in the second it's us
who has the binoculars note that the
constituent parts in both scenarios are
exactly the same but the syntactic rules
used to recombine them produce different
meanings compositionality is therefore
incredibly important because it leads to
open endedness the ability to construct
an arbitrarily large number of
meaningful complex expressions from a
finite number of constituent parts hence
it's incredibly important that our
representation
should also be able to support
compositionality to summarize evidence
from neuroscience suggests that
representations should be compositional
have nice untangled manifolds support
efficient and adaptive attention in
class 3 and have the ability to
incorporate leads in information another
related discipline that can inform our
search for what makes a good
representation is physics this is
because physics constrains not only the
development of biological intelligence
but also the form that natural tasks
which we are interested in solving can
take general bias a representation to
reflect certain fundamental physical
properties to make it generally useful
for solving many natural tasks it
appears that there does exist a
fundamental idea that underlies pretty
much all of modern physics that that we
may be able to use to help us think
about representation that is the idea of
symmetries as was nicely summarized by
Philip Anderson a nobel laureate in
physics it's only slightly overstating
the case to say that physics is the
study of symmetry so what is a symmetry
to a physicist
symmetry is a broader concept than the
reflective form of butterfly wings
symmetry represents those stubborn cores
that remain unaltered even under
transformations that could change them
let's unpack that using an example of a
simple spring mass system you can unroll
the system forward in time to produce
the bonds or you can translate the
system in place the two transformations
can be applied interchangeably without
affecting the final state of the system
this commutative property of the two
transformations
means that one is the symmetry of the
other an idea formally captured by Emily
Nothe in 1918 which also related
symmetry transformations to the notion
of conserved properties the stubborn
course about
world mm-hmm ever since the publication
of north us first theorem symmetries
were used to unify existing theories and
physics for example electricity
magnetism could have been unified based
on their shared symmetries to categorize
physical objects and even to predict the
existence of yet undiscovered physical
objects like was done with the Omega
minus particle that was discovered two
years after being predicted from the
existence of a gap in the taxonomy of
symmetries the same idea applies more
abstractly in the case of natural tusks
to for example 3d scenes can be
transformed by changing the scale of an
object or its position and the two
transformations can be interchanged
without affecting the final state of the
scene and hence can be seen as
symmetries of each other so physics gave
us a hint that our representation should
somehow reflect the structure of
symmetry transformations in order to be
good now let's turn back to AI and see
what ideas from the machine learning
toolbox can be utilized to build
representations that meet some of the
desiderata we obtain from the other
disciplines as mentioned at the start of
this talk the majority of success
stories in modern deep machine learning
come from some form of supervised
learning one way to analyze what these
deep neural networks are doing is
through the information bottom line
perspective which suggests that the goal
of supervised learning is to find a
maximally a compressed mapping of the
input variable that preserves as much as
possible the information on the output
variable what this means is that if we
were to have any hope of solving the
task Y then the input data X should have
at least all the information about the
task as well as potentially
other unrelated information this has to
be true since due to the data processing
inequality no information can be added
through post processing processing it
can only be reduced then the goal of the
layer wise processing in deep neural
networks is to discard all the nuisance
informations unnecessary to solve the
task to find the maximally compressed
representation in the last layer this is
an example of invariant representation
learning which aims to map the results
of any transformation G that does not
affect the task or on the input space
through the same value in the output
space for example to classify an object
we don't need to know where in the image
it is located so we want to learn a
representation that is invariant to
translations a key idea behind the
hugely successful conversion neural
networks an alternative idea is that of
an Ecuadorean representation learning
where the goal is to preserve the
effects of the transformation G in the
representation space so that if the
input is translated the representation
is also transformed in an accurate in an
equivalent manner one example of such
equivalent representation learning is
captured by stranded work called
disentangled representation learning
while no agreed-upon definition yet
exists of what disentangling is a common
intuition assumes a factorized
generative process from a set of
semantically meaningful latent variables
for example a simple data set of 2d
sprites may be fully specified by the
position size rotation shape and color
generative factors this entangled
representation learning aims to invert
the generative process and learn to
infer a latent representation z hat
that matches the generative factors from
simply observing the entangled images
note that the idea of disentangling in
machine learning is conceptually closely
related to the idea of untangling in
neuroscience
despite being developed independently of
each other hence closing a loop with
neuroscience closing the loop with
physics the idea of disentangling was
also recently related to learning
representations that preserve
information about symmetry
transformations while the formal
connection requires some basic
understanding of group theory which
unfortunately we don't have time to go
to into today an intuitive explanation
goes as follows let's assume a set of
symmetry transformations that we want to
capture in our representation for
example these can be horizontal and
vertical translations and changes in
color in a simple grid world let's also
assume that these transformations affect
the abstract straight state of our grid
world indicated by W here independently
of each other so if we apply the
horizontal translation transformation it
only changes the value of the
corresponding abstract world state
coordinate X let's also assume that
there is some generative process that
map's the abstract world States to the
observations that our AI receives for
example pixel images of a sprite on the
canvas the goal of our AI system is then
to learn a mapping from this observation
o to representations that such that a
function f which is the composition
about generative and inferences
processes be an H respectively is an
Ecuadorean map this means that we want F
to be such that we can either apply our
transformation in the abstract space W
and then go through or first go through
and then apply
the transformation in their
representations but space said either
way ending up in the same state in our
representation if we can find such a map
F then our representation is said to be
reflective of the underlying symmetry
transformations the important thing to
note is that such an Ecuadorean map
would naturally split the representation
space space that into Independence of
spaces each one of which can be
transformed by the corresponding
symmetry like change in positional color
independent independently of the other
subspaces so having scavenged physics
and neuroscience for inspiration about
what makes a good representation and
having established some connections with
useful machine learning ideas to gain
traction traction in fulfilling these
desiderata
how can we verify that we are indeed on
the correct path before plunging into
years of research trying to come up with
an algorithm that would actually learn
third presentation we want let's use the
example of our proposed symmetry
preserving representation and go through
the potential verification steps first
reasoning about how much it fulfills our
list of the strata from physics and
neuroscience and then looking into
whether such a representation could
possibly help address some of the
outstanding shortcomings of deep
learning by definition our
representation reflects symmetries and
through connections to disentangling
this representation is also untangled
furthermore since the underlying
symmetry group is meant to be a
composition of subgroups like changes in
position and color our resulting
representation should also be
compositional since the representation
is split into independent subspaces it
should be able to easily support
attention for example implemented as a
binary mask over the learn
sucks faces in all the support
clustering we need not only attention
but also a meaningful metric to judge
similarity this can easily be achieved
by showing that our representation lives
in the vector space and picking any of
the multiple metrics that are defined on
vector spaces to judge similarity forced
a clustering hence it appears that our
proposed representation does tick all
the boxes in terms of the
multidisciplinary insights we gained
today now let's see if our
representation may also help address
some of the shortcomings of the current
deep supervised and reinforcement
learning systems mentioned early in this
talk at least in theory let's first look
at a deficiency it's likely that the
majority of natural tasks would be
specified in terms of some of the
entities conserved by symmetry
transformations which in turn would
correspond to the independence of spaces
recovered in our representation for
example solving natural tasks like
naming the color of an object would
require a simple transformation of a
single learned independence of space
corresponding to color transformations
indeed a number of papers including the
deep symmetry networks cited above have
already demonstrated that incorporating
symmetries into deep neural networks
does help reduce the date efficiency
supervised tasks since our mapping else
needs to be equivalent to many natural
transformations by construction its
functional form will probably have to be
quite constrained which is likely to
make it more robust to other serial
noisy perturbations evidence suggests
that this may be the case and it has
been provided by recent work by Cole and
colleagues furthermore augmenting our
representation with attention is likely
to help with robustness too as was
recently shown by
Zorin and colleagues the attention based
model was shown to be much harder to
understand fool than the baselines as
shown in the example on the slide in
order to miss classify the image of a
wallet as a beaver the adversarial
attack actually had to draw a shadowy
beer image in the top right corner of
the image for the new attention based
model while hardly perceptible noise was
sufficient to fool the state-of-the-art
resonant bass bass line that didn't
include attention as you may remember
from our venture into neuroscience
generalization can be increased if the
decision on which action to take can be
made without taking into account those
aspects of the representation that are
not important to the task
since our symmetry based representation
set preserves the important information
about the stable course of the world in
a format that allows for fast
attentional attenuation we can quickly
adapt the minimal set of informative
subspaces available to the decision
network when faced with solving ever
starts tasks thus increasing the
translation of such decision networks
a similar story goes for transfer since
our mapping app connects the underlying
abstract symmetry transformations to the
representation it should not care about
the nature of the intermediate
observation indeed it was shown that the
schema networks a model that use hand
engineered features reminiscent of our
symmetry covariant subspaces could
transfer its ability to play break out
two noble variations much better than
the unstructured deep reinforcement
learning baseline finally even though
this is perhaps the least explored area
of improvement for machine learning some
preliminary evidence suggests that our
hypothesized representations may support
compositional abstract imagination and
may be a solution for grounding
many promising discreet or simple based
algorithms which have been shown to
display some of the desirable properties
means missing in deep neural networks
like concept induction or abstract
reasoning while unfortunately no
algorithmic solution currently exists
for learning such symmetry equivalent
representations in a robust and scalable
manner walking through the verification
reasoning steps suggests that aiming for
such representations may indeed be a
promising research direction in
unsupervised representation learning to
conclude this part of the talk let's
recap we have seen that designing good
representations played a crucial
historical role in machine learning this
early role of explicit representation
learning was made obsolete by the
successes of end-to-end deep learning
which seemed to be perfectly capable of
finding good representations implicitly
these algorithms however still have many
shortcomings that are becoming more and
more prominent as we start reaching a
plateau in exploiting the strengths of
the current systems many of the current
advances in addressing some of these
shortcomings have already been
attributed to learning back to
representations for example by adding
axillary losses or inductive biases to
the models this suggests that further
advances may be accelerated if we start
thinking about what makes good
representation explicitly and try
biasing models to learn such
representations intentionally one way to
gain insight into what may make a good
representation is by looking into
related disciplines like neuroscience
cognitive science physics or mathematics
hence given the prominent past and
potential future role of good
representations in machine learning one
next wonder is all machine learning
ultimately about representation learning
on this note I'm going to pass the baton
to Michaela
who will tell you about some of the
existing methods that have made first
steps on the path towards learning good
representations hello everyone I'm mija
I'm a research engineer at deep mind and
a PhD student at UCL and today I want to
talk to you a bit about a few
representational learning techniques
every now has talked to you about the
kind of properties that we expect and we
desired from our representations and now
we're going to talk about how we can
achieve this kind of representations
using deep neural networks and the
approaches we're going to talk to about
today focus on three main pillars and
this pillar sometimes blend together but
they roughly are splitted as follows
we're going to talk first about
generative modeling the kind of
representation learning techniques which
learn representations by modeling the
underlying data distribution of our data
set we're then going to talk about
contrastive losses which use
classification losses to learn the kind
of representations that preserve
temporal or spatial data consistency and
then we're going to move on to self
supervision and in still supervision
we're going to use information about the
data modality are we talking about
images are we talking about audio and so
on to build the kind of representations
that to learn to exploit some sort of
information about this data so for
example if we crop an image we want to
learn very similar representations to
the original image and so on and as we
go through these types of representation
learning techniques we're going to try
to evaluate them so how are we going to
achieve that well we're gonna first
start looking at semi-supervised
learning so all of the methods that
we're going to talk about today are
using only unsupervised data to learn
representations so we have our
unsupervised data set without any labels
and then we're going to learn our
representations but then we have to ask
the question well how good are these
representations
answering other type of questions about
the data in parts types of these
questions that we might have as well
what kind of object is present in the
data
and so on and in order to do that we
often build downstream tasks to try to
assess what kind of information is in
our representation so for example we
might want to train a classifier on top
of our representations and ask the
questions well how much of the
information in the label is present into
our representation and often we do that
by putting a very simple classifier
something like a very simple linear
layer to just make sure that we answer
the question what kind of data is
present in in our representation and as
we do this we often have in mind the
concepts of data efficiency we want to
learn the kind of representations that
are efficient at then being able to
answer this kind of supervised asks with
you labels but we also want to be able
to generalize so we want this kind of
representations that when we ask a
different question we're still able to
use them for that additional question
and one standard benchmark that is used
for semi-supervised learning is imagenet
so Imogen is a data set of large
resolution images containing a thousand
different labels cats dogs and so on but
what we do in representation learning is
first train our representations on image
net without any kind of label
information and then we use perhaps only
a small percentage of our labels to
train a classifier and then see how well
our representations are doing and this
approach allows us to compare different
types of representations but we also
want to learn representations for
reinforcement learning in reinforcement
learning we have an agent that acts in
an environment and often we want to have
agents that acts for example in multiple
different environments or we want agents
that quickly adapt with experience so we
have the kind of task that is very hard
to achieve using a lot of online data so
for example if you want to train a robot
to pick up an object
training such a robot takes a very long
time in practice so we might want to use
simulation and transfer from simulation
to reality and we will see that learning
representations or learning disentangle
representations for example can aid the
speed of learning or the ability of
model to transfer for simulation to
reality and we'll also look at model
analysis so this from the perspective of
our talk is mainly for us to understand
what the model is doing when in practice
we also want to see what kind of
representations we have learned Rd
satisfying the kind of properties that
yuna has talked to you about this
entanglement and so on and they are also
useful from the perspective of learning
interpretable models we want to see what
the model r is learning especially
before deploying it in production for
example and as we go through keep in
mind that we want both discrete and
continuous representations because the
underlying data that we model often has
discrete datum factors or continuous one
so for example if we try to describe a
face whether someone has glasses or not
is a binary variable is true or not but
someone's hair color for example is
going to be a continuous variable is
going to be something that goes from one
spectrum to another and so on we'll also
want to learn representations that adapt
with experience both from purpose the
perspective of reinforcement learning
but also from the perspective of
continual learning and other types of
approaches that we want to see for
classifications for example we want to
be able to learn quickly and adapt our
representations as we go through we also
want to learn representations that
contain the kind of consistency that we
expect in our data so for example if
you're trying to learn a representation
of a scene we want that representation
to encode so much information about the
scene that we can answer questions such
as well how would this scene look like
from a different angle perhaps an angle
that the model has never seen before and
the same goes for temporal abstraction
that we might want for text or audio and
so on and always when we talk about
downstream tasks we want to think about
the per for
ones that are representational learning
approaches bring to our downstream tasks
but also the data efficiency aspect so
how much faster are we able to learn
that now that we have integrated our
representation learning methods so let's
start with our first type of approach
given by generative modeling so what is
generative modeling and why is it useful
for representational learning well in
generative modeling we're trying to
answer the question what kind of
distribution could have generated our
data set and remember we're starting
with purely unsupervised data so we have
our data set here shown in green we have
our points in this case they are split
in two different modes and we're trying
to answer the question well what
distribution could have generated this
data and here one potential answer is
this mixture of two gaussians here and
the natural connection between
representational learning in general of
modeling is first given by the
theoretical aspect and that has to do
with compression learning probability
distributions efficiently has a lot of
theoretical connections with being able
to compress data and we want to learn
representations that are efficient and
compressed in a small number of bits but
also intuitively if I'm trying to model
this probability distribution that's
generating this data we know that
modeling this cluster here closely
together is efficient as efficient as we
would want for example model this
cluster of points here together and this
is a very small one the example but for
example if we thing here but they
doesn't like image net that has cats and
dogs and so on we would want to learn
the kind of representation that foster
cats together in clusters dogs together
and and so and there's a specific type
of variable model that allows us to
model the kind of representations that
we would like and these are latent
variable models so we've talked about
this probability distribution that could
have generated our data but often we are
also interested in modeling the
generative model of
the data so being able to model the
sampling process how can i generate new
data points from this distribution and
we can either model P of X and then
sample from this distribution or we can
model the generative process directly
and this is one example in this case of
latent variable models we assume that
the generative process looks like this
we have our high dimensional data so for
example images in high resolution and so
on and we assume that these are
generated by applying a very complicated
mapping which we often model using a
deep neural network from a low
dimensional space so we have a few
attributes in low dimensional space
these are going to be our discrete and
continuous representations and we're
going to learn to map that into our
dimensional space and intuitively if we
think for example of faces those are
very very high dimensional if we take a
high-resolution image of a face but
there are few underlying factors that
determine how that face look like the
background color whether this person has
glasses or not and so on but from the
purpose of representational learning
we're interested in inverting this
process so we're interested in answering
the the question well given a particular
data point
what kind of representations or what
kind of latent variables could have
generated this data point and this is
our posterior latent variable or
posterior distribution over latent
variables given our data point so we're
no longer just looking at one
representation but we're looking at
distributions over representations and
the key trick is that in practice we
will learn this jointly so we will learn
to do generation and inference together
and also important to note is that we do
not have access to the ground truth of
the latent variables we are doing
unsupervised learning so we're trying to
learn what the latent variables are as
we learn how to generate and how to do
inference crucially learning this
posterior distribution P of Z given X is
often intractable so we might have to
resort to certain approximations and
just to make this very concrete if we
have our generation process we have our
set of
latent variables some discreet some
continuous and they can generate a face
for example of a face of me here behind
a pig pink background and we have
different kind of representations and
our goal is to learn these
representations and to do inference so
keep in a particular image to answer the
question well what kind of
representations could have generated
this image purely unsupervised so no one
will tell us well this person is wearing
glasses and so on
and one way to achieve this is to use
maximum likelihood or to start with
maximum likelihood so maximum likelihood
is a very common training criteria for
genitive models because it's very
intuitive what it's saying is well given
our date data set I want to train a
model that assigns high likelihood to
the data so this expectation we won't be
able to compute exactly because we don't
have access to the true data
distribution if we start but we have
samples from it so we can use Monte
Carlo estimation but when we look at the
right hand side here we see that well
what this is saying is that when you
take a sample from the data distribution
our model has to assign high likelihood
so this is very intuitive we want the
model that is able to explain our data
the challenge becomes when we're
starting to look at latent variable
models and training them with maximum
likelihood when we're looking at latent
variable models this P theta of X so
these are our parameters of our model
that were aiming to learn is now given
by an integral so why is that well what
is the weight that the model assigns to
a particular data point X well that
weight is given by the probability that
a particular latent variable generated
that X remember in our generation
process from latent to observe data
times the probability that we see that
latent variable summed over all possible
it and variables that we have and here
we've introduced this probability of a
particular latent variable which is our
prior this is something that we can
choose and it's very related to
obtaining the kind of representations
that you would want that I have for
example this entanglement or
other properties that Irina talked about
so how do we get around the fact that
now we have a logarithm of an integral
and in practice for complex models that
becomes intractable so we cannot compute
that in closed form well what we can do
is instead use a bound on the maximum
likelihood objective something that is
often called the evidence lower bound so
if we look at this law of probability
distribution here this is what we ideal
you want to optimize in the maximum
likelihood objective but we know that we
don't have access to be C dot of X so
instead of maximizing this we maximize
the bound this is the bound here and
we're trying to make the bound as close
as possible to our original objective
one thing to notice here is that now we
have a new object in our bound that
looks a lot like what we were talking
about before to be our posterior
distribution so now we have this
distribution with learned parameters
data that looks at the posterior
probability of seeing a particular
latent variable given X but this is just
an approximate posterior for complex
models we will not be able to compute
the posterior exactly and this is what
makes the difference between variational
inference and expectation expectation
maximization you might have seen
expectation maximization or e/m before
and in M you actually do this
alternation between expectations and
maximization and there in the first step
you actually compute the true posterior
and when you have the two posterior this
inequality here becomes an equality so
you're actually maximizing the original
maximum likelihood objective in the case
of complex models like the ones we're
going to talk about you cannot compute
this posterior exactly but what we will
do is we will make sure that the
posterior also maximizes the evidence
lower bound so this whole objective so
that we are as close as possible to to
the true likelihood so now we know what
the approximate posterior here is it's
trying to approach
made the true posterior P of Z given X
that we don't have access to but let's
look at the individual terms of Te elbow
so the first term is called the
likelihood term and what it does it's
trying to ensure that we're encoding in
the latent variables as much information
about the data and the way it does that
is by saying well I'm having my data
point X that I started with I'm passing
it through my posterior and now I have a
posterior or an approximate posterior of
latent variables given X and now I want
to sample from that posterior this is
what the expectation says and I'm
passing these samples from the posterior
to our model and this model now has to
assign high probability to the original
point that we've seen and in order to do
that we have to have latent variables
that could explain X our data point in
our generation process and the only way
we can do that is we if we somehow have
been birthdate our generation process in
our in our inference network which is
what we wanted to achieve to encode
information about our data point in to
set on the other hand the second term
says something quite different it says I
want the approximate posterior to be
close to the prior and this is
specifically important from the
representation learning perspective
because the prior is something that we
choose so it's a knob that we can tune
so that we have our approximate
posterior to have the right kind of
property so for example this
entanglement if we choose our prior to
be disentangled such as for example a
Gaussian distribution that has
independent by mentions this KL term is
going to regularize our approximate
posterior to also have disentangled
representations and if we're thinking
about combining this approach with
neural networks that's when we obtain
variational auto-encoders and the key
behind variational auto-encoders is that
both our approximate posterior and our
model P theta of X given set our model
using deep narrow metrics so in order to
get the parameters of our posterior we
pass our
data through our encoder and now we get
for example if we use a Gaussian as we
often do in practice we get the mean and
the covariance or the diagonal of the
covariance of the Gaussian distribution
so we've talked about the importance of
the Cale term from the perspective of
regularizing our representations but if
we know we want this entangled
representations for example what we
might want to do is increase the weight
of the scale we can either and nearly
during training or doing from do it from
the beginning and so on and while this
is no longer an exact bound on our
maximum likelihood estimate it tells the
model that I really want to learn this
entangled representations so you should
also encode as much information about
the data as you can into our
representations but these
representations should be close to the
prior and this allows us to learn
disentangle representations and this is
what we can empirically test by going
through the model analysis aspect that I
was talking about before and this is
what's called a latent reversal so
imagine we have a set of say seven
latent variables and we keep them all
fixed apart from one so all six let's
say we have our first six latent
variable are fixed and we change our
seven latent variable and for each value
of the seventh eighth and variable let's
say from minus 3 to 3 we make a small
grid we ask the question well how does
the data generate it change if I just
change the seventh eighth and variable
what does this tell us well it will tell
us what the seventh latent variable is
encoding because all the other ones are
fixed so we're going to for example
analyze here what happens in the case of
a bit of EE and we see that in this case
the seven latent variable encodes what
object is in the scene as we go from the
value minus three of the latent variable
to the value three the only thing that
changes in the scene is the object and
we can do the same for other types of so
for other indices of our latent variable
so for example latent variable for
encodes the distance of the
object to the camera and so on however
we can look at what would happen if we
don't train a bit of a so if we train a
standard V the kind of representation
that we will learn will probably be
entangled so for example here is an
example and we see that if we change the
first latent variable both the proximity
of the object as well as the background
are changing and so on and if we go back
to our point of looking at using
representation learning techniques for
downstream past such as reinforcement
learning we want to assess how good
these tasks are at how good this
representation techniques are at for
example transfer in reinforcement
learning and this is especially useful
in robotics and what we empirically see
is that learning this kind of
disentangle representations of scenes
using a beta be like model allows
reinforcement learning agents such as
Darla's that are using beta is to
transfer quicker from simulation to
reality so far we've talked about fees
like models that are doing inference and
generation in one step but if you ask me
to draw a car I'm not just gonna draw a
car just like that I'm gonna start with
an outline and I'm gonna keep iterating
on that and this is the idea behind
control the idea here is to start any
iterations both in our generations and
our inference process we're still using
a VM we still have a reconstruction
likelihood term and a care loss but we
are adding this recurrent component and
if we look at how the generation process
looks like in control we see that we're
starting to gradually move from very
high outline to seeing that there's
going to be a car in the image to more
detailed view of a car and so on up to
the final fine-grained view of the car
and this is the generation perspective
but from the inference perspective this
allows us to build powerful posteriors
so so far we've talked about masses that
use this type of Gaussian posterior for
example but by being able to have latent
variables
that are both spatial and temporal we
can have posterior distributions or
approximate posterior distributions that
are autoregressive so the latent
variables at time point to depend on
latent variables at time point one and
so on and this allows us to build
complex posterior distributions that can
be closer to the true posterior that we
don't know and we don't have access to
which makes the original bound tighter
so we are closer to training to the
original maximum likelihood objective
that we started with another approach
that uses time warm layers to learn
representations is morning so in one day
in the author you authors with both beta
bees and attention networks to learn
both through segments completely
unsupervised the objects in an image and
to learn the zentangle representations
of these objects so how is this achieved
well we start with our original image
and now we pass that image to an
attention network which decides what to
focus on at this particular time step in
the learning process so at this time
step the model has decide to focus on
all backgrounds here apart from the the
sky so the the wall and the floor and
then this only this input the mast input
is provided to a beta P that learns how
to reconstruct that part of the image
but we're also keeping track in a scope
of everything that we have learned how
to reconstruct so far and thus at the
next time step the model knows that
we've already focused on that part of
the image and chooses to focus on
another part such for example as the
cone and now the bit of a learns how to
reconstruct only that part of the image
in the Florence representations for that
specific part of the image and as we go
through we keep building and building up
the scene by focusing on different
aspects at the different time points and
also learning these disentangle
representations of team of the objects
and we can also test this again by doing
a similar type of latent reversal that
we talked about before and looking at
for example the blue object
and how Layton's how the scene changes
if we change the latent variable for a
particular aspect of that object so for
example we will again look at the first
latent variable change it and see what
kind of semantic information changes in
in the scene and here we see that for T
position of for the blue objects we have
a one latent variable that encodes the x
position and we have the same for the
red object and we also have a latent
variable that includes the Y position
and so on and having access to a model
that learns both how to focus on the
right objects in a scene but also
learning the same tango representations
of those objects it's very important for
reinforcement learning often
reinforcement learning tasks focus on
objects that actor or the agent has to
pick up something into ended in the
scene and move it somewhere else there's
also distractors objects and so on and
what we want to see is that if we learn
this kind of representations that focus
on objects and have this idea of what
could be present in a scene that we are
able to transfer quicker from certain
training tasks to take testing tasks and
this is what can empirically be seen
with money here we've moved from one
distractor to two distractors and the
agent is able to quickly learn how to
perform in in the test environment so
far we've talked about general methods
for representational learning without
necessarily looking at the kind of
consistency properties that we might
want so if we want to learn a
representation of a scene for example we
want that representation to encode
information about how the scene would
look like from different angles and we
can't train a model to encode that by
providing the model with different views
of the scene at different angles and
telling the model what the angles are
using neural network to encode that
information into a neural scene
representation and using that as
conditioning information for a
generative model very similar to draw
the model that we've seen before that he
uses multiple generation steps to
generate a final predicted view
crucially given a particular query so
now the model has to learn that if I'm
providing him with a different query
angle here you need to predict a
different view and so on and the
information about the scene has to be
encoded into our neural scene
representation vector because the whole
model is learned into and so the
reconstruction loss that the modeling
curse here is back propagated through
the rendering steps and back propagated
to the neural scene representation such
that this representation encodes all the
kind of information that the model would
need to be able to answer this question
and if we look at how gqn is able to
answer these questions well we can first
see how we can provide the model with
different images at different views of
the scene and how then the model is try
to answering just trying to answer the
question well how would this look like
at a different angle and again it does
this in a temporal fashion so it keeps
iterating upon the scene and improving
and improving until it gets to a point
where it's very close to the ground
truth so here if we just focus on this
last two columns we can see the
prediction here and the ground truth and
we can see that they are very similar so
the representation has encoded enough
information about the scene that it can
answer these questions how is the scene
looking like from different angles that
it hasn't seen before and crucially gqn
is also able to capture uncertainty if
we provide it with very little
information about how the scene look
like looks like so for example we just
say well there's a wall here we then see
that it's able to imagine that there
could be multiple objects behind the
wall it's not just generating the same
object again and again it knows that
well there could be a blue ball or a
blue cube or multiple both at the same
time being behind the the wall and this
is a kind of very important property
that we would want from our
representations we want them to encode
uncertain
about the world that they are seeing and
this kind of representation of a scene
is of course very useful and
representation learning but also being
useful for the downstream tasks that we
want to perform so such as reinforcement
learning so you mention again that we
have this art that is trying to pick up
an object in the scene now we can
compare different reinforcement learning
agents at this task the ones that are
learning directly from pixels show here
in orange and the ones that use gqn
representations that have this
representation inbuilt that know to
answer this kind of information about
the scene hold the scene look like from
a different angle and we first see first
the importance of the data efficiency
aspect that I was talking about before
when learning from pixel on a simpler
task where the camera that the agencies
is fixed the difference between gqn and
not using gqn and learning from pixels
is the difference between learning very
quickly and with low variance so here
all the agents are able to learn versus
learning slower and some agents don't
pick up from from the grounds that
they're not able to learn a task however
if we move to a harder task where the
camera is are moving so the agent has to
adapt to different views of the scene
around it we see that gqn makes the
difference between being able to learn
the representations and being able to
learn the task and not being able to
perform the task at all so here we're
seeing the importance that
representation learning can have in
reinforcement learning so so far we've
talked about BAE based models that use
this type of likelihood based
reconstruction losses with continuously
done variables so all the approaches
that we've talked about either
sequential or not use continuous
representations but we might also want
to learn discrete representation now the
challenge here is that when we learn
these models we want to learn them end
to end using back propagation just like
I've shown in the gqn case
but if we have discrete latent variables
in the middle we have a sampling process
of a discrete random variable so back
propagating through that sounding
operation is very challenging because it
has high variance so we can use
estimator such as we enforce for example
but it makes it harder for us to to
learn one trick to get around using this
these estimators is to use the discrete
latent variables as indices into a
learnt embedding space and this is what
the UPA does so for example we start
with our image just like we would do in
a San Rafi case being coded using our
encoder and now we have a continuous
vector of representations of that image
what do we do with this vector well
we're looking into a learn table of
embeddings and we're looking for the
nearest neighbors into this table and
the indices of our own vector elements
into this nearest into this table are
going to give us our discrete variables
which encodes the data once we have our
lookup and we have our continuous values
given by the table we can take that into
our decoder get the data and back
propagate all the way through using a
straight row estimator so now we're able
to learn discrete random variables in
using this kind of approach and what
they'll also show is that they're able
to achieve very high compression rates
so when we're thinking about
representation learning we're also
thinking of the number of bits that we
need to encode these representations of
the data and what the author shows that
with these discrete representations you
can achieve a very high compression
ratio use only a very small number of
bits to encode the data and you still
achieve good reconstruction so for
example these are a few reconstructions
obtained by PQ PA this is the original
image and this is the reconstruction but
one thing that you might notice is that
the reconstruction are a little bit
blurry and this is what happens when you
use this kind of likelihood based models
like des it's not specific to v QV all
the other models presented so far well
this type of reconstruction but not all
latent variable generative models use
this type of likelihood base loss and
actually there's a very popular type of
model called generative adversarial
networks that does not use this loss and
what this model does it it learns a
latent variable model to a two-player
game
so this two-player game more accessible
we have a discriminator that is given
real data and samples from our model and
has to distinguish between them so it
has the answer the question well is this
particular image real or is this
generated on the other hand the latent
variable model the generator has to map
from this noise space just like we've
seen before through a deep neural
network and generate the data such that
the discriminator can no longer tell
whether this is real or generated so we
have this alternating approach between
well between our discriminator to learn
how to distinguish between real and
generated data then we improve our
generating generator using signal from
this discriminator then we improve our
discriminator based on this improve
generator and so on but what we've
described so far has a little terrible
model it has a generator that maps from
latent variables to data but we don't
know how to answer the inference
questions so given a particular image
what kind of representations should I
use for this image so we're not using
reconstruction losses we're training our
model using an adversarial approach but
we don't know how to do inference and
there multiple ways to do inference
using this ganon style
losses but one that we're going to talk
about today is given by by can or a big
bike and a scaled version of PI again
and here the authors showed that you can
learn how to encode information about
the data by changing your adversarial
game so now we've added a new component
to our game and this is an encoder this
is very similar to what we've seen in
the VA case we have a data point we pass
its word people internet or again we get
our encoding of our data and we have our
generator there just like in the gun
case or in the case that the examples
from the prior
Bassett's with a generator and we
obtained a sample from our model now the
crucial change is how the discriminator
changes so so far we've seen that the
discriminator distinguishes between
samples from the data so this is X and
samples from our model X hat and this
makes sure that through training the
generator is matching X and x hat so
it's matching the data distribution and
the generated sample distribution but
now we want to go beyond that so now we
no longer only want to match the data
and the samples we also want to match
our latent variables using a prior and
crucially to invert this generation
process from from samples to pronate on
samples to model samples and this is
what our inference would do we start
from data and we go back to
representations and the way we can do
that is to have a discriminator then now
distinguishes between pairs so we have
our data point X and we have its
encoding given by the encoder so into a
forecast or encoder we get our
representation set hat and the model has
to distinguish between this pair and a
pair of samples latent variable samples
from the prior and the generated image
given by the generator given this latent
sample so now this joint distributions
have to be matched so what does this
mean it means that firstly the marginals
are going to be matched so the model is
gonna learn how to generate high-quality
data the latent variable distributions
are going to be matched so for example
the marginal distribution of our
encodings is going to be equal to the
one of the prior just like in the ve
case but now we've also matched the
relationship between X and set hat and
set NX so this means that now that hat
is encoding information about X because
Zed is generating X hat so we're able to
invert this process using this two
player game and distribution matching
and crucially we're not using any
reconstruction laws so just with this
adversarial game were able to learn how
to reconcile and how to do
representation learning and crucially
because we don't have picks the level
reconstruction losses our wreck
instructions look very different so
remember we've seen the vqv a wreck
instructions they were pixel based train
so we had this likelihood losses that
we've seen in the VA case but here we
don't have that so what the model
chooses to do when it tries to represent
image that looks like this so winter
scenery with someone with a red jacket
and a hat and a few trees in the
background
is to focus on the high-level
information present in the scene so
there's still a person here someone
wearing a red coat a hat there's still a
lot of trees and there's a winter
scenery but the pixel level
reconstruction laws between these two is
quite high so this is because the model
is not using a likelihood based approach
like we've seen with with the models so
far and this also tells us that the
latent variables are encoding high-level
semantic information about the scene so
the latent variable is encoding that
well there's someone with a red hat in
the image rather than focusing on
particular pixel values and at the time
of publishing this this type of
representation learning methods that
encodes this high-level information by
this adversarial game was
state-of-the-art on this image nets semi
supervised classification tasks so where
we take our representations and we put a
layer on top of them to do
classification a linear layer and ask
well how good are our representations at
learning this downstream classification
task so so far we focus on latent
variable models but not all generative
models that are useful for
representational learning need to be
generative more models that use latent
variables so for example we can use auto
regressive models of text train using
maximum likelihood such as GPT to learn
useful representations that can be used
for downstream tasks such as reading
comprehension
attention translation summarization
question answering and so on and the key
here is to use a very well tuned neural
architecture with billions of parameters
and large amounts of data so the data
said that they used was very large and
very cleverly cute
so the take-home message so far focuses
on this idea of using latent variable
models as a powerful tool for
representational learning and we can
train this latent variable models using
maximum likelihood adversarial training
or other approaches but when we're
treating generative models we're asking
the model to do something very difficult
we're asking to learn a probability
distribution a high dimensional space
and to generate data in this high
dimensional space and one natural
question that we might have is well can
we get away from having to do generative
modeling to be able to do representation
learning and this is what contrastive
learning tries to do it still uses
completely unsupervised data just like
all the approaches that we're going to
talk about to learn representations but
it removes the need for a generative
model and it uses a classification of
laws instead but the classification loss
is built from the unsupervised data and
done such that the right context in the
data is encoded into our representations
so let's look at this completely one
example of contrastive losses and how to
use that for representation learning is
for Tyvek so work avec leur Nations of
text and this is specifically important
for text because if I'm trying to encode
information about an image if I if I use
the pixel level records representations
we still have meaningful information
encoded in the pixels so similar colors
are going to have similar pixel values
and so on this is not the case for text
so if I get a dictionary and I try to
input that dictionary into my model the
simplest approach is to use one what
encodings so what does that mean well
I'm gonna think the first word and the
representation that I'm gonna use for it
is going to be 1 0 0 0 0 0 and so on the
second word is going to be 0 1 0 0 0 0
and so on this encodes absolutely no
semantic information whatsoever so then
if we try to look at these
representations and for examples know
that China is to Beijing what Russia is
to Moscow watch a pan is to Tokyo and so
on we won't be able to do that this one
in coatings do not provide any of this
kind of information so we're trying to
learn representations of texts that
encode semantic information and we can
do that by learning a model which
predicts the kind of representation it
should expect given the past context
given a few words that it seems so far
the kind of representation that it would
expect at a future time step and the
really crucial bit about this type of
loss is that we provide both positive
and negative examples so we're training
the model and we're saying this is what
you should expect in the next time point
but this is also the kind of words that
you should not expect in the next time
point and using this kind of approach
the more the authors are able to show
that you are able to encode semantic
information also using a very clever
testing approach so for example if we
think of translation and representations
of words in different languages the word
representations should be an relatively
similar and especially the relationship
between words should be similar because
they all encode this kind of underlying
structure of the world we live in so we
would want to test whether
representations such as word effect
representations are encoding this kind
of information and the way to do that is
to say well I'm going to train the work
effect model completely unsupervised on
English I'm going to train a word to
back model completely unsupervised on
Spanish and then I'm going to use a few
examples to learn a simple linear
mapping between these representations
using supervised data so very few
supervised words so for example saying
that well this word in English is this
word in Spanish and so on and the
crucial moment comes when you ask well
is this mapping generalizing so have we
learned a way to do dictionary
translation just using a very simple
mapping if we use the right
representations for our words and the
answer is yes so if you swore to vaca to
learn the useful representations you can
then translate between English and
using a very simple mapping so this
gives us confidence of the kind of
representations that were to buy castle
earned so for example if we look at bet
is the word bed in English is translated
as commas in Spanish and the dictionary
entry is exactly the same and even for
those which are not the same the
semantic meaning is very similar so for
example this is the case for the word
prayer which is Spanish azorath illness
and while the dictionary entries results
there's still quite similar in terms of
semantic meaning encoded in the word
another approach that uses this type of
contrastive loss by providing positive
and negative examples of the right
context is given by contrast of
predictive coding or CBC and this
approach tries to mix maximize the
mutual information between our data and
the learn representations so here again
we're providing the model with a
positive example of the right context at
future time steps and a batch containing
that example and a few other negative
examples of contexts that is not in the
right the right place so let's look at
this from the architectural perspective
so we have our in this case sequence of
audio and we want to learn
representations of of this audience by
splitting it into chunks so we're
learning a mapping from audio chunk to
are encoding Z and we're doing this by
sharing a neural network between each of
the time points so we want to our aim is
to learn this encoder so how are we
gonna do that well we're going to encode
oral all our time steps that we've seen
so far up to some point xD let's say
using our encoder so we're gonna have
set t minus 1 set T and so on and then
we're going to use an auto regressive
model to encode all that information
into one vector and that vector should
include all the information about the
sequence that we've seen so far and if
it does that that means that it should
be able to answer questions well what
kind of
datian do you expect in two time points
because we know that for example if the
audio is seen so far is an audio group
for example it shouldn't expect a song
and so on and this can be done again
using this idea of providing positive
examples and negative examples of the
kind of context at the model it should
expect and this can be done by choosing
data from from our data set so given our
audio data set we know what will come in
two time points so this will will
provide us the positive example but we
can also take for example the chunk that
will come in four time points and say
well this is not something that you
should expect in two time points because
it's not at the right context so the
crucial bit here is to think of this
idea of temporal coherence and this is
what the model is trying to enforce the
kind of representations that know the
temporal coherence structure of in this
case images but this approach can also
be done for spatial data such as images
so for example we can look at the
spatial relationship between different
patches and the data and we can use that
to learn representations and what we
what we see is that the model is able to
do what we want from our
representational learning techniques
which is to do very well in the
circumstance when we don't have a lot of
labels so we're looking here at the
image net benchmark of this
semi-supervised learning benchmark where
we're asking the model to predict a
particular label from D representations
that it's learned and here we start with
80% fewer labels than in the original
data set and we compare the model train
on CPC using CBC representations with
the one train from pixels so this is
what we would do for example if we don't
use representational learning methods
and we see that in the regime when we
don't have a lot of label data the model
that uses CPC that uses the
representational learning technique
performs a lot better
another new approach that uses this type
of contrast of laws to maximize
agreement as mutual information is
simply R so in seem clear the approach
taken is somewhat different but very
very interesting so we start with our
original data point X let's say this is
an image of a dog and then we're trying
to answer the question well if I
transform this image a little bit I note
that I want to still learn the kind of
representation that encode the
high-level structure in this image so
imagine that first I start with my image
of a dog then i zoom in a little bit on
the top this is my first transformation
the second transformation is going to be
that I rotate to D and talked a little
bit it's still an image of a dog just
from a different angle and a different
perspective I take both of these images
and map it through the same function f
that gives us my that gives us the
underlying representations that we want
to learn so we know that this
representation should still contain the
kind of information that we had in the
original image it should not be exactly
the same between the two views because
we still have transformed the data so we
don't want it to fully be the exact same
representation but we want that if we
pass our representations to a different
mapping we obtain some variables then do
look very similar indeed because these
mapping is is in to extract the
underlying information that we started
with so the fact that the image encodes
there's a dog behind the green
background and so on and then when we
want to take our representations and use
them from downstream tasks what we do is
we take the output of the first mapping
so the mapping that still encodes the
information that we have a dog but
allows us a little bit of different
structure between the different
transformations and the kind of
transformations that we have for
different data modalities will depend
but here the author's are showing us
what we can do for images so for example
we can crop and resize we can add
Gaussian noise Gaussian blurs begin to
color distortions and so on
using this kind of approach the authors
are currently as far as I know state of
the art on this image net benchmark by
adding a linear classifier on top of the
Train representations and specifically
they own have a plot in which they
control for the number of parameters in
our model so for example if we look at a
hundred million parameters we see big by
gun that we've seen before
so the model that uses adversarial
learning to learn representations and we
see that sing clear at the same number
of parameters performs substantially
better if we look at four hundred
million parameters who are talking about
larger models we see CBC also model that
we've seen before and we see that seemed
clear outperforms that as well so so far
we've seen this idea of using
classification losses to encode this
kind of temporal or spatial consistency
into our representations but see clear
also shows this very powerful thought of
using information about the data
modality and this kind of
transformations that we can have about
the data modality to learn our
representations and this is the idea
behind self supervised learning the idea
here is that we will design tasks often
classification or regression tax so
supervised tasks again starting with
completely unsupervised data such that
we encode the kind of information that
we would expect in our representations
and the key kind of tasks that we would
want are those that for which it's easy
to obtain data but will encode and will
allow us to exploit as much as we know
about the data modality so for example
image polarization we start with a data
set of colorful images which is easy to
obtain again we don't require any labels
just our data set of images and we use a
standard tool to turn these images to
black and white so now what we ask the
model is to revert to this mapping so we
ask the model to learn how to colorize
and if one does this right then we can
use this again for representational
learning and we can use this for
semi-supervised learning in imagenet
just like we've seen before with the
downstream tasks for classification so
crucially here we started with an
unsupervised data set and we created
supervision from that unsupervised data
set by thinking of the kind of
properties that we would want our
representations to encode so to know the
kind of colors for example that objects
could have but we can go beyond colors
so for example what we can do is we can
take an image and ask our model to learn
spatial consistency again by looking at
the relationship between different
patches in the image so for example if
we have a centered image here which
would be the center of the face of the
cat as shown here we also can ask the
model well given a particular patch one
of these eight patches which one do you
think this patch is right so this is the
ear here shown on the right and what the
model has to learn is well if we have a
cat at the center and we see a year if
the ear is at this particular angle it's
on the left or right side of the cats so
now the model only has to do eight-way
classification but in order to be able
to answer this question accurately it
really has to understand and learn the
kind of representations that are useful
for semi-supervised learning or object
discovery because it has to learn how
objects related to for example other
objects or two different views of the
objects and so on we can also go beyond
the images and look at sequences so for
example if we have a few frames of
videos we can then shuffle them and we
can ask the model to sort the sequence
and to run to learn the kind of temporal
coherence that we want from the data
that for example in order to hit a golf
ball you first have to hit the ball and
then the ball will fly in so on and you
can do this without ever asking the
model to predict
your frames so the model is not
predicting how the frame would look like
it just has to tell the order this is
the first frame this is the second frame
three a third frame and so on and these
kind of representations are then used
also for downstream tasks such as object
recognition action recognition as well
because the model has to learn the kind
of temporal coherence that is required
to understand what what's an action and
so on and one example that combines a
few of the methods that we've seen
before is pert so burnt learns
representations of text by leveraging
both tasks that allow the model to learn
local structure and global structure so
what do I mean by that one of the tasks
that Byrd has to do is to learn what
objects what particular words have been
masked in a input sentence so for
example given a sentence we can mask a
few words and then the model is asked to
say well what words have been masked
what what is the right word to do here
and this is reminiscent of some
approaches that we've seen before
crucially the models also ask to answer
questions such that given sentence a and
sentence B which one do you think will
come first so this is long term temporal
coherence and long term structure which
is very different than the kind of
representation that you need to answer
the local structure of this word comes
here and this word comes here and so on
and word also combines this clever task
design with this idea of using the
advances in the neural network
literature by using directional
transformers to train the model and
Trudy Byrd has sparked a revolution in
natural language processing it's used
for multiple downstream tasks
summarization named entity recognition
spam detection and so on and it's been
also put in production as part of Google
search by improving the quality of
search results so self supervised
learning is a different kind of approach
that allows us to use this domain
knowledge that we have about the data to
build useful
for presentation learning so we know
that if we have a sequence of videos we
want to have a model that knows how the
frame ordering should look like and we
can train a model to learn that and then
use those representations for downstream
tasks and we can go beyond video images
text and so on so this is a very
powerful approach that goes beyond one
particular domain so as you will go
through and read more about
representation keep in mind a few things
so tasks design is very important for a
presentational learning not only for the
self supervising learning so supervised
learning case but also for contrastive
losses and so on data modality is very
important we've seen that you can use
the kind of different transformations
for images or different kind of
approaches for texts and so on so if you
want to learn representations of images
you'll probably use a different approach
if you want to use the representational
learning for text context is important
and we've seen this with gqn we're
trying to learn representations of
scenes and answering the question well
how would the scene look like from this
angle and hold the scene look like from
this angle and so on also generative
models are very useful for
representation learning and we've
specifically looked a lot at latent
variable models but you might be able to
get away without asking the model to be
able to generate data at the high
modality so for example things at high
resolution for images and so on and you
might be able to use this type of
cleverly designed classification losses
such as those obtaining contrastive
losses and self supervision without
having to use generative models and
we've seen again and again the crucial
benefits of incorporating hiral
architecture so we've seen this with
Monet that used attention to learn
concepts in a particular scene we've
seen this with bird that uses the
transformer and we've seen this again
and again of using even the thought of
using model such as resonance and
convolutional networks is very important
so what is the future of representation
learning so I hope I've convinced you
that a lot of progress has been done
just in the last few years it's amazing
to see the rapid progress in the field
but there's still more to be done in
terms of Jarett of model approaches we
can build powerful posteriors and better
priors so we've seen the effect of
posteriors and methods such as draw that
use auto regressive posteriors we've
talked about the importance of priors
when we discuss this entanglement and we
looked at the latent reversals of beta V
s versus Vav es and we can push that
further for new approaches for
contrastive learning perhaps we can go
beyond temporal and spatial coherence
and for some self supervised learning
this is just the beginning and the kind
of tasks that we can design by
exploiting the kind of information that
we have about the data modality is very
very important and we can do a lot more
in the future and this idea of
incorporating changes in neural
representations is very important so we
will see that the field of deep learning
will advance at the same time as the
field of representational learning
incorporating each changes back in
representational learning is very
important and last but not least the
field is starting to think a lot more
about causality we're starting to talk
about causal coherence so what would
have happened had I done this and having
the kind of representations that are
able to answer this questions are going
to be very useful for a lot of
downstream tasks such as the ones that
we've talked about so far for example in
reinforcement learning and that's it for
me thank you very much for your time and
I hope you enjoyed the lecture
you