Deep Learning 6: Deep Learning for NLP

I'll get started so just some
housekeeping before I start and this is
just a sanity check so I'm told it's a
two-hour lecture so we're done by 4:00
is that correct and you're it's probably
expecting a break of like 10 minutes or
so about halfway through okay
so we'll try and make that happen after
at some point tonight the slides will be
put on whatever your internal system is
called my Moodle Moodle it's cute okay
and this year I I'm not privy to what is
on the exam so please don't ask me you
don't have an exam okay maybe that's why
I was like I didn't ask me to write exam
questions this year okay then that
preempts this all the awkward questions
about it's my code I need to memorize
work direct for the exam cool so I med
graph instead normally Tori's here to
introduce me but apparently is cramming
on some paper the and I'm a research
scientist at deep mind I'm also an
honorary faculty member in the machine
reading group here so you may see me at
other lectures and I've been asked to
give a lecture on deep learning for
natural language processing today if for
some reason you were at the lecture I
gave last year in this course which you
probably wouldn't be you can leave
because it's the exact same lecture now
I tweaked a few sites but and it's a
difficult lecture to write because we
have entire courses centered around this
title at Oxford Cambridge Stanford and
I'll give you some links at the end of
the slides if you want to delve into the
topic in more depth there are excellent
lectures but deep learning for NLP more
than just a sort of subject or
application of these techniques is a
very rich field indeed so there's a few
options about what I could do in a
lecture and one of them is try and wrap
through eight weeks of slides in two
hours but that would be somewhat
unpalatable and that I don't know how to
wrap or I could just give you a sort of
a little taster and that's more what
I've done in this lecture
language is fundamentally interesting
problem why because animals some animals
have some primitive forms of language
but it's one of the sort of key
characteristics that distinguishes us
from other forms of life and it's one of
the tools we use to formulate
abstractions and communicate them so
more than just a way of you know getting
things done on a day to day basis it's
actually part of our cognitive toolkit
it's how we think it's how we talk to
ourselves it's how we make plans so it's
integrated into the higher form of
intelligence that we sort of Express so
it makes sense that if we are walking
towards wanting to make artificial
general intelligence or AI that is
similar in ways to humans that we'd want
to sort of solve language and I'm not
gonna solve language for you today
because we are very much at the sort of
start for steps in this in this space
but I'm gonna cover a few topics to give
you a taste of some of the topic some of
these sort of questions that are being
addressed for which we have models and
where research is going so I'll start it
off by talking a little bit about like
how we can build representations for
words because they are in many respects
the sort of common building blocks of
our languages although obviously we
sometimes want to deal with characters
or diagrams I'll talk a bit about
language modeling so kind of the
unsupervised learning of of an natural
language understanding and talk a bit
about conditional language modeling and
how we can use language modeling
objectives to learning translation
models which will bring me to sequence
the sequence models with attention and
all these things are building on things
that either you've seen previous
lectures in this course or have yet to
see because I don't remember exactly the
order of the topics I'll talk a bit
about a few of the applications of these
methods to natural language processing
and understanding and end with a few
slides on composition and classification
so let's start by talking about word
vectors which is for some people a
fascinating topic but certainly not my
favorite so I'll try and simulate
enthusiasm but it's important to
understand it so in natural language we
you went to another language
understanding and processing we want to
be able to deal with text typically and
that might come from a speech to text
synthesis or from like the vast amounts
of X we see on the internet and text is
a sequence of discrete symbols so words
or characters or ideograms in other
languages so we need to build a
representation for this text if we want
to feed it into our models especially in
your networks which expect vectors as
input so the very naive representation
of a word if you wanted to sort of be as
unbiased in general as possible would be
simply to have a vector space which is
the size of your vocabulary and each
word is a one hot vector in this space
so for example you you assigned to every
unique token and index and you said what
and to produce a vector for it you
produce you put a one vector you put a
one at that index position and the
vector and everything else zero so this
might sound really naive but it's in
fact fairly successful in very simple
systems so classical information
retrieval which sort of I guess over the
years evolved into technologies like
Google's used very much use this
representation to do document retrieval
so in classical information retrieval
the document and the query vectors are
just going to be super positions of word
vectors so you want to produce a
representation for each webpage you
simply sum all the vectors of all the
words in the webpage and that gives you
a single vector the size of the
vocabulary with trivially the counts of
each word that occurs in the document
likewise your query you can treat as a
small document which you're going to
execute against your retrieval system
and then you take some similarity metric
amongst the vectors like they're in a
product or at the cosine distance and
you return the documents or the K
documents that give you the highest
score and that's information to
retrieval in like a nutshell some way
for like well we're classification
problems like document classification
you can do very naive sort of count
based metrics along this this dimension
so the obvious issues are that this is a
very naive sort of representation so
there's nothing in the representation
that tells you about similarity about
semantics about relations
so humans we understand that cat and
kitten are different words but that
those can't the concepts they point to
have some sort of notion of similarity
but in this sort of representation they
are literally mathematically orthogonal
so they're sparse orthogonal
representations they have no semantics
and obviously we can do better so we
want richer representations that Express
semantic similarity and before I jump
into how we do this with neural networks
or these sort of vector-based mm you
know machine learning models I want to
give you a bit of the context for how
these models came about and so there's
an idea that was postulated by several
thinkers in the 50s
including the philosopher Vidkun Stein
in some fashion called distributional
semantics and it's the most famous quote
in a relation to this idea of semantics
is this quote by Jennifer you should
know a word by the company keeps the
idea is to understand words we sort of
look at how they're used what sort of
other words occur around them and this
gives us some information about their
syntactic and semantic roles so the
three main approaches to distributional
semantics that involve producing dense
effective representations based on the
context or use of words rather than
these very sparse orthogonal
representations we have by taking one
hot vectors
there's count based methods and we'll
start by describing those there's
predictive methods which are sort of
proto neural networks but mostly by
linear models and then there's task
based or end-to-end sort of approaches
to learning representations and I'll
sort of briefly mention those towards
the end of this section so count based
methods again are probably not something
you're going to be one thing to
implement because we have better ways of
doing this but it's important to
formulate its formal a high level
intuitions about what you know what word
vectors are and why we build them the
way we build them by looking at how
people use do that up until probably the
late 2000s so for companies methods
you're gonna start by defining a basis
vocabulary see of context words so this
could be all the words in your
vocabulary if you're particularly
ambitious and have a lot of memory but
typically you're going to want C to be
the sort of the salient context words
that you expect to be informative so
people have a variety of heuristics for
this you take maybe the top 10,000 or
top 5,000 words in the language maybe
exclude stop words like gun and because
those don't tell us really that much
about like the words they co-occur with
other than syntactic markers but so the
first type of parameter is basically
your context vocabulary the second sort
of hyper parameter for this approach is
a you define a word window size W so
under the most naive approach we're
going to consider the context to be W
words on sort of like three words on
either side of your target word or four
words or five words and obviously the
size of this window is quite meaningful
in terms of the representations you're
going to be producing and once you've
defined those two things you simply to
count the number of times each basis
vocabulary words occurs W words to the
left or the right of each instance of a
target word for which you're building a
representation so context words might
also be target words for another part of
your model but the target words are the
one are the words you're building
representations for at the moment so you
would rate through your your target
words that you want vectors for and you
simply collect counts like this and you
form a vector representation of the
target word based on these counts so to
give you a little illustration let's
imagine we have a little snapshot of the
corpus and we have target words kitten
cat and dog that might occur one or more
times so we collect here three words on
either side and let's say that we have a
small basis vocabulary based on our
prior knowledge of what sort of what
we're looking for which is the words a
bit cute furry loud meow per gram smell
small so we don't include things like
and/or though and so we simply first
collect for each target word the context
words so we see that kitten has cute
perch small meow cat has cute furry
meiotic cetera and we can just turn
those into vectors by turning putting
the counts as ones or the actual count
value which happened to be one for
convenience at every position in the
vector that we mapped that context index
to so sorry I feel like that was ever
ever said so obviously each context
where it has a unique index and then we
set the count for that index in the
vector to
the count and zero if it doesn't if it
if it doesn't occur in the context so
that just gives us these representations
and now you can see in contrast with one
hot representations kitten and cat
obviously had some overlapping ones as
do cat and dog so once we have collected
these vectors so we can use similar any
sort of distance metric like the inner
product or cosine distance as a
similarity kernel for the concepts that
the word stand for so we can check that
now for example using cosine similarity
as a distance metric we can check that
the similarity between kitten is cat is
point fifty eight and the similarity
between kid and dog is zero and the
similarity between cat and dog is point
twenty nine and these numbers don't like
have a probabilistic interpretation or
anything particularly strong except we
know but they're bounded between zero
and one but what we observe here is that
we now have through this very trivial
account based methods the notion of the
semantic continuum and that it's not the
case that you know everything is only
identical to itself but some concepts
are more identical to others and
obviously this is you know handcrafted
to match our intuitions but if this
tells us just from looking at words the
Kerkar that kitten and cat are probably
closer more closely related concepts
than dog and cat or dog and kitten
respectively just based on looking at
how they're used so as a quick reminder
cosine similarity obviously is nice
because it's its norm invariant so it
doesn't matter how if some words might
be very frequent and they have very high
counts and some more it's my being a
frequent have low counts but we're by
using cosine we can compute similarity
by just looking if they're pointing in
the same direction so that's it's a nice
Auto normalizing measure so what's wrong
with count based methods well there's a
lot of reasons why we've gone beyond
them the first is that not all features
are equal so first we have to
distinguish counts that are high because
the words are particularly informative
from counts that are high because the
words are independently sort of frequent
in context so some words for example
like that's a good example like red or
or small are going to be very frequent
across
variety of descriptors of animals for
example so they might not be super
informative compared to sort of like
freckled or sort of carnivorous those
might also be frequent but you get what
I mean right so some words some words
just happen in a lot of context so
aren't very good at distinguishing
particular concepts some words are going
to be very identifying a particular
concepts and obviously this depends on
your task so it's difficult to sort of
have a nice ordering of those so there's
normalization methods that allow us to
sort of like post process or count based
vectors in order to sort of try and
capture that idea that some things are
going to be some more some context words
are very informative in your core and
your corpus and that they allow you to
distinguish concepts and some just
become part of the background noise and
should maybe should have been removed
from the context in the first place like
and or or which we heuristic ly remove
so we can take like the term frequency
divided by inverse document frequency as
a common normalization method there's
point-wise mutual information and I
won't go through describing these in
details because you can just look it up
and survey papers if you're very
interested but it's good to see that
there are some methods that try and sort
of capture this sort of semantic
contribution of context words and some
are in fact quite sophisticated that
they they remove the need for a norman
variants emulator similarity metrics by
turning the vectors into probabilities
but there are preps like easier ways of
sort of addressing this or a problem so
one of the easier ways to deal with the
problem of having to pick your features
so by picking your context words very
carefully and thinking about how to
renormalize them is to not have to do
that so instead of having a very
carefully hand engineered feature space
which is effectively what we're doing
with traditional distributional
semantics we can do what we do well on
deep learning and neural methods and
just function approximation in general
and jointly learn the fee and just learn
the features from from the data so
before I jump into how that works let me
give you the intuition of whom an
embedding matrix was some of you if
you've implemented anything involving
text will be familiar with as an
operation so that all those vectors that
we collected in that previous slide can
be sort of stacked into a matrix since
they're all the same length or the
length of the number of context features
I've defined right
and this matrix is what we'd call an
embedding matrix we call this an
embedding matrix because if I take the
one hot vector for a word right which is
that sparse representation we started
with assuming that the index so the
index at which it's 1 corresponds to the
row it's ad in this matrix then trevally
by taking the multiplication of the one
hot better by the embedding matrix I
retrieve that dense vector that we
collected counts for right so here I've
separately done the sort of live
separately have pre learned my embedding
matrix through the count based method
but there's a simple idea that there's a
differentiable operation that map's a
sparse discrete representation of the
token of the symbol to a dense
representation of its semantics whoops
and all the methods were going to be
seeing from here on are going to involve
instead of using count based methods and
heuristics to define this embedding
matrix we're going to learn it through
back propagation so symbols are unique
vectors and representations are sort of
the embedding of those symbols within
our semantic space using this embedding
matrix are there any questions so far
yes so where does that have multiple
senses are an interesting and opening
problem so obvious the approaches you'll
see in natural language processing with
neural nets even more sophisticated ones
assume that there's a single embedding
for the word and that you're gonna have
to deal with that so I mean if you want
to there are probably more sophisticated
ways if you have external information
like for example you've done a sensitive
word sense disambiguation step by using
a part of a pre trained pipeline like
Stanford Cornell P right you could
assume that you're gonna pre-purchase
you're gonna do pre-processing and
you're going to build separate
representation for each senses of the
word with that and that involves the
input of a particular bias because
you're using the bias presented by your
tool into your model another way you
might look at it is to say okay I'm
gonna have a single representation for
the for the symbol that's a dense
semantic representation and I'm going to
trust that my model can represent the
the totality of the senses in that
vector and obviously that means that if
you use the vector on its own you're not
to be it you're not going to be able to
disentangle those different sentences
census directly but if you're inputting
that vector into let's say recurrent
neural network or a big or a comp net or
wave medics that are things that can
capture long range dependencies and you
know effect information flow maybe the
previous context will allow you to sub
select parts of the vector relevant to
one sense versus the other so for
example if you have Bank right which
might be a river bank or might be a
financial institution and you're doing
translation right and the previous ten
words were like I went down to the white
no that doesn't design because it's like
I took my I took my wallet and went down
to the bank when you read Bank there
because of the presence of wallet in
your context maybe a network could be
sufficiently intelligent through a good
training objective to learn to gate part
of the representation relevant
riverbanks out and only include the part
relating to financial institutions but
that's always putting a lot of trust in
your model so maybe there's a more
clever way to do that by modeling using
lane variables like the plurality of
sentences so it's a great question that
it's open research yes yeah because so
if you're for example if you're doing
counts count base vectors are nice
because we can directly interpret the
features and you can think about these
issues in an idealized context and
obviously if we have several represent
if we have several if we don't have many
context words right some context words
are going which might be informative
with regard to distinguishing Bank from
let's say train might not be
representative might might not be useful
for distinguishing both sensitive back
and Bank internally because they occur
in both right in which case suddenly you
have a useless feature when you have a
useful feature with regard to like
separating words that have different
tokens but you've got different types
but you have a useless feature with
regard to this is a big raishin task and
indeed the solution might be I need more
and more context features but in free
increasing the context window size also
include like includes noise right so you
trade-off one problem and create another
which is the difficulty with not just
that approach but also the neural nets
so it'sit's that it's definitely an open
problem is how to deal with a sense
plurality or ambiguity any other
questions great so let's talk a bit
about how we can learn this embedding
matrix so neural betting methods and in
general and when we've seen neural
embedding methods we're including things
like word Tyvek which is not per se a
neural network it's a it's a log by
linear model but neural network is taken
today to occur to pretty much anything
with a matrix and back prop so don't if
someone super pedantic you know I don't
care so the general method and obviously
there are approaches that don't fit this
mold kind of fits this pattern we're
going to collect instances of a word
that we want to build a representation
for in our vocab and so obviously this
is this is our target word as seen
previously and we might have many target
words we're training in parallel but
let's just treat it in somewhat in
isolation for now and for each instance
we're gonna collect its context words
and so context words here could be
something like we did before which is
like using a key word window where again
the size of the making the window large
might you know give you a more
informative context might add more noise
making it too small will remove noise
but make it your listen formative or you
might do something smarter like say it's
gonna cry it's gonna be dependent on
syntactic information like parse or just
you know I'll limited to sentence
boundaries or if you're using more sort
of modern models like recurrent neural
networks you might just say you know
I'll just I'll just figure it out as
we're going along and you don't you
don't need to explicitly specify the
window so but for attribute for the
methods we're gonna be looking at now
this windows usually explicit and we're
gonna different we're gonna find some
scoring function that's going to be a
function of the token word its context
words so the words that occur with it
and some parameters which typically will
have an upper bound on the output so
that we want to we want to maximize it
and we're gonna find a loss which is
usually which is going to be across all
the words that we
to build representations for and across
all the instances of those words the
negative the some of the negative score
so Amit by minimizing my max minimizing
this we maximize the score across all
the words we're going to build
representations for and this is nicely
linearly separable we're going to
estimate then through usually a gradient
descent based method the optimal
parameters of whatever at the scoring
function is which we might not reuse we
might just be training that for the
purpose of training or embedding matrix
and the embedding matrix itself and
we're going to use that estimated e as
our embedding matrix so before I jump
into some examples the scoring the
choice of your scoring function in this
context which is going to be the
objective against which you're going to
be building you're going to be building
your vectors is very important
especially in these sort of more and
design methods we'll be looking at
unless when you're doing tasks based
embeddings so it's easy to design a
useless score so and you can easily an
accidentally design a score that ignores
the output or that ignorant sorry that
ignores the input or that doesn't have a
bound on the output and I mean the
trivial example of this is if you're
using a score that allows all the
embeddings to go to zero and then
maximizes your minimizes your loss then
you you you you create a useless score
and this was the sort of trap that a lot
of people fell in in the very early days
of neural networks or function
approximation based embeddings so the
ideal desiderata for your scoring
function here is that it has to
obviously embed your target word using
the embedding matrix because if it
doesn't then what are we back propagated
how are you updating the embedding
matrix through back propagation it needs
to produce a score which conceptually of
these tracks a function of how well the
that target word is accounted for by its
context so how well the context explains
the presence of that context word or
vice versa how well the presence of that
target word explains the context and
we'll see some examples of this and we
want and ideally we would like that we
assume that this that we're training
this on on natural text on text that we
obtained from humans ideally the scoring
encodes the requirement that that word
is the best word that could be there so
that by replacing it by random other
word should lower the score that the the
scoring function of reflects that that
word belongs there more than any other
word in your vocabulary so these are all
like strange assumptions but we'll we'll
see how they translate into reality and
into particular scoring functions and
finally this should produce a loss which
is differentiable with regard to the
parameters of the scoring function and
the embedding function otherwise you're
gonna have a hard time training it's
your back propagation and this is
probably not the sort of thing where you
want to jump into reinforcement learning
so one of the early approaches to
embedding by using a neural network is
neural embedding models by cold britain
Western Western so this is a really
beautifully titled paper filled there's
a natural language understanding from
scratch or natural English processing
from scratch from scratch thing is just
beautiful beautiful marketing and in
court yeah and it's a it's a cool paper
and you so this method is not something
you'd use these days but this really I
think like started a revolution in
natural language processing by making
people realize maybe we should be
looking into these neural networks again
obviously people were looking at neural
networks and like the 70s and 80s and
natural language processing but this
this maybe people think let's do it
again and most of the authors working on
this are now sort of quite senior
researchers in Facebook AI research or
in deep mind including Corey lilyc who
is the head of the departing group okay
I'm pretty sure is on that paper
so the approach here is simple we're
going to and we're gonna start by taking
our embedding matrix which again is just
a matrix you're using to project one-hot
vectors into dense embeddings and we're
gonna embed all the words in the
sentence so we're gonna produce vectors
for I'm assuming that you can see and
usually like the point I can point so
you're going to produce vectors for each
word in the sentence using your
embedding matrix so that's the first
differentiable step you're going to then
use a function that you know is a
shallow convolution so a function that
you can repeat because you might have
different length sentences is going to
convolve those into a single
representation by producing doing a
projection across
like three or four or five or W words
and project and then summing all those
particular applications of that
convolutional map so that's like a
little compliment if you will and that
produces representation then you might
apply non-linearity and you then project
those into a scale or using an MLP of
however many layers you want there is no
point in memorizing these architectures
by the way this is just to try and like
all these are for the purpose of giving
you an intuition about how embeddings
are now this is fine because you don't
have an exam but this is usually where
people ask me just like are you gonna am
I gonna have to repeat these specific
details of this in the exam and that
would be pointless information for the
examiners so that that's really the
whole idea is like barf out some vectors
using your embedding matrix project them
into a single vector map that vector
into a scalar which is your score and
the overall network is there for
modeling just a func scoring function
over the sentence so if you if you were
just at that particular if you stopped
there and you said okay let's let's use
that scalar directly as our score and I
want to let's imagine we're treating as
a loss for example and I want to
minimize that loss what could I do I
could just for example I set my
embedding matrix to just be a matrix
full of zeros right because then that
would if I if I if that's like the
lowest score I've defined that will just
trivially minimizing my loss but I won't
produce anything particularly useful so
they have to produce a scoring function
or an actual loss based on the scoring
function that prevents the network from
ignoring the output or producing sort of
degenerate or trivial solutions because
we want the vectors that we produce here
to be informative so remember I told you
one of the desiderata for these scoring
functions is that they sort of you know
show that that that word accounts for
the context that replacing it with a
random other word from that each word
accounts for the context in that
replacing it with a random other word
should be less good according to the
scoring function and so how they how
they transform that thought into a
differentiable loss is that during
training for each sentence they're going
to sample
random corruption of that sentence by
replacing a ward at random so we have
our true sentence s and we have our
distractor sentence which is identical
except that one word is random Zed and
you're going to minimize the hinge loss
which is good by taking the difference
between the represent the set of the
sentence with its distractor minus 1
because we want to maximize the
difference yes
but up to a point so the him she says
it's like there's no there's no point in
making them two different you just at
least one that you want to maximize it
within bounds and so this means that if
you do something trivial like map
everything to zero and then these are
going to be identical and so then your
loss is going to be one and you can't
minimize it further so you there needs
to be some difference in the
representations that yields a difference
in the score and this is kind of you can
read this right okay there needs to be
some difference in the representations
leading to a difference in the score
that's significant enough that this gap
is closed as close to one and then if
it's if they're significantly different
by for example having a model that's
getting a little too excited making
vectors like maximum the orthogonal then
you don't get any benefit thanks to the
hinge loss objective so they still need
to be vaguely similar because they're
there they are very similar sentences so
you just have what a corrupted word so
the interpretation here is that
representations that you're learning in
your embedding matrix are carrying
information about what neighboring
representations should look like so this
means that if you're like changing a
word then something the model should be
able to capture that there's that
representation doesn't belong there that
that according to the words that I'd
have seen that some word stands out and
I can tell you I can tell you which one
at least to the point of giving a lower
score than if that word had been the
correct one so by producing by setting
this objective which seems really simple
right it's just basically have a four
word function that projects everything
into a score and to find a hinge loss
we're producing something that looks a
lot like the distributional hypothesis
that the sort of low energy
configuration that gives you a sensible
scores means that the vectors contain
information about what their
neighborhood should be like so you know
worried by the company it keeps
literally but this is the the Niroula
fication of that idea
it's a sensible model because it kind of
fits a high-level intuitions about what
we were doing in distributional
semantics but it's fairly deep and this
is deep by the standards of the day
obviously this stuff's pretty trivial to
run under GPU these days but it's fairly
deep compared to some other methods so
it wasn't cheap to train and the
convolutions that you use to produce
that representation are going to capture
very local information so that's kind of
like your that's like a window size
hyper parameter and it's very sensitive
to that so another approach which is
still popular which people still use and
continue to re-implement and tutorials
is is work Tyvek so weird effect is in
fact two models skipped Graham and
contextual bag-of-words and how am i
doing for time okay well so the approach
here is is much simpler in significantly
more GPU friendly if you care about
those things so the start we're gonna
embed context words and we're gonna add
them and we are going to project that
vector that we've had from simply adding
all our context words in our window
sorry got distracted by that bottle so
step one contextual bag of words we're
going to take a window around the target
word that we want to build a
representation for and it removed that
target word so we're gonna ignore it for
a second we're gonna embed the context
words so all the words except the word I
care about and add their representations
into vector so that's a very simple
operation and then we're going to
project that representation back into
the size of the vocabulary and do a
softmax operation so that is going to
essentially produce a distribution over
you know what the context word could be
what the removed word could be given the
context so we have a very clear
probabilistic interpretation of this and
on top of that the fact that we're just
taking a sum and then projecting means
that we don't need the that we can move
that we can do little optimizations and
move the sum inside the softmax re do
the embedding operation once and then
project after doing the sum which
is nice and efficient and the objective
here doesn't need any sort of very
careful hand crafting it's just
minimizing the negative log likelihood
of the correct word so we just take the
cross entropy with regard to the correct
word that was held out of that
distribution so it's nice and so we
often in language talked about negative
log likelihood because it's it's just
you know minimizing the negative lock
the hug likelihood maximizes the log
likelihood which also maximizes the
probability of the correct label so
these things are all equivalent but you
get to do it in log space which is to
build a stable so this is nice so it's a
nice model it's all linear so it's very
fast and it's a very cheap way of
playing just one matrix to all the
inputs so rather than having to sort of
take the sum of the projections and then
doing the softmax historically how this
was trained was by doing negative
sampling instead of doing the softmax
directly but today on modern computers
and with libraries like tensor flow you
can have a fast unstable softmax on GPU
and use that instead of any fancier
training mechanism so actually
minimizing the negative log likelihood
directly is the preferred option and
it's much more stable there's a bunch of
variants that have been explored to this
idea of a contextual bag-of-words so for
example weighing laying and kris diary
2015 instead of just taking the sum of
the context vectors transformed the
transform the context vectors by a
matrix depending on where it is
positionally with regard to the target
word and that can be seen as like weakly
capturing possibly slightly syntactic or
word ordering information and there's
like two or three years worth of ACL
papers on the topic of word embeddings
doing it more in a more sophisticated
way which are all basically variants on
this approach the dual to the contextual
bag of words where the context needed to
predict the word is skip gram and this
is possibly the more popular variant in
or Tyvek where now you are going to
produce an embedding of the word that
you're trying to build a representation
for projected into the target vocabulary
and predict all the target words
in the in the context so that's
literally the whole idea is and I take
your word embed it so I take your word
embed it and then project to the
vocabulary that gives a distribution
over words that might be in the context
take the part law the negative log
likelihood of the context words is going
to be the negative log of the product of
the probability of each context words
given the target word that you've
observed and that nicely goes out to
some so you can basically just do it in
one pass and read the logics off off
your off your single projection so it's
very efficient to implement and that's
possibly the main reason why it's
preferred so superfast approach there's
just one embedding operation when
projection read the probabilities from
the softmax and there are similar
variants to contextual bag-of-words
so conditioning on the position rather
than just producing the probabilities of
the target words willy-nilly but
obviously you're always adding
computation adding parameters to do
these so in general and these approaches
we have we see a nice sort of trade-off
between the efficiency of the method and
a more structured notion of context
which I guess is the easy way of
summarizing all the work that happened
after your after work two vectors let's
add more computation and more structure
and get better representations but it
might a add more bias and be certainly
as one computation so as a as an
engineer or as a model designer you need
to you need to pick one based on those
constraints so that's sort of summarize
we've sort of seen a I'll take questions
in a second but we first seen a quick
history of word embedding methods more
from a sort of linguistic perspective
count based methods and we've seen how
those can be turned into sort of
efficient little and neural networks or
function approximated to remove the need
to like think about what are my context
words what are my stop words and what
the data just pick those things for you
so you lose interpretability in that the
dimensions of the vectors no longer mean
it's like oh I've seen this context word
this many times but on the flip side you
don't need to make that decision so win
some lose some
any questions on this part of the
lecture yes
that's a good question
probably so people have like worked on
applying convolutional nets to like
prediction assert that a variety of
approaches include machine translation
and generally sequence modeling sequence
labeling and the sequence
classifications particularly on the
cones group at NYU now cash Brenner the
people behind wave net whether people
have saw not aware of any individual
work that has tried to do exactly what
you mentioned but it sounds like
something that I probably heard someone
talk about
so unfortunately showing my lack of
expertise by not being able to point you
to a paper but if no one has done it
please do do it because it sounds like a
sensible way to try and interpret a
neural network yes yeah so I mean so
relying on dependency trees was a very
popular approach when we were using
count based methods because you're
constraining something to you're only
counting as part of the context things
that had some syntactic relation which
you'd hoped would Express the semantic
relation so a lot of work in the 90s did
use this kind of approach and the sort
of more neural approaches to just
weren't embedding learning I think you
see a slight improvement by conditioning
on word position sewing legs paper kind
of shows this but it's not so amazing
that people care about it a lot I mean
people are focused on incorporating
syntactic information at a higher level
when you're building sentence
representations and often that's
sufficient to to learn words extrinsic
just from like the end objective yes I
should have mentioned I'll get back to
it later in the lecture but the third
approach to learning word embeddings is
to say I have an embedding matrix as
part of my general model whether it's
sequence to sequence or sequence
classification or all I have an
objective like learning to translate or
learning to classify sentiment from a
sentence and I just back propagate into
my embeddings and learn them from
scratch instead of pre training your
embeddings you can just do that I think
I'll probably repeat that later on but I
hope that answers your question for now
any other questions
dependency tree is a form of parse tree
so for example you can translate CFG
parse trees into dependency trees which
are DAGs expressing that something is
like the direct object of the verb
versus like the subject so just another
way of expressing syntax a syntactic
relations in the sentence
okay so let's jump into discrete
language models and then we'll take a
break so we've seen words let's move on
to how we can deal with larger use of
text so have you had a lecture on
language modeling already with her Anand
okay great
you'll probably see this again at some
point in like either/or we all given a
lecture yet did you talk about language
modeling you did a little bit okay well
revisit this it's like hearing things
twice is great you'll be like to
remember it okay so language modeling is
we've talked about like scoring
sentences based on like whether it
belongs in it or not that language
modeling is a significantly more
powerful objective and that is
effectively what we're doing in
unsupervised learning which is you have
some data and you model the probability
of your data so you want to you want to
it for unseen sentences or unseen data
points or sequences or unseen images
unsupervised learning which gives you an
explicit density model should be able to
say okay here's the probability of this
data point according to the distribution
of data I've seen during training so
that's exactly what language modeling is
about it's about estimating the direct
probability of a sequence of symbols by
learning to maximize the probability of
a corpus of symbols so the pret this
probability of a sequence is usually
decomposed left to right but obviously
it's mathematically correct also do you
compose it right to left and you might
want to do this if you're doing language
modeling for say Hebrew so you decompose
it left to right as the product of the
probability of the next token given all
the tokens you've seen so far which we
usually write in some sort of compact
notation like this so this sort of
underscore one two I minus one means all
those tokens from index 1 to I minus 1
so in practice what we're trying to do
with the neural network and/or with
other methods in language modeling is we
want to estimate this conditional
probability with the probability of the
next token given a possibly unbounded
prefix of tokens is according to the
data we have so Before we jump into
neural networks the most popular
approach to language modeling in fact
the state of the art until probably two
or three years ago tops was Engram
language models so the Engram language
models try to address this issue the
probability we've just defined depends
on a variable length unbounded history
of our sequence and how do we model that
because we might have different lengths
and we want to be able to we can't just
for example record every particular
instance and and just pretend that we
have a variable length probability
distribution in a table so one solution
if we want to use tabular methods is to
introduce an ordered K Markov assumption
so in order to K Markov us a Markov
assumption in sequence modeling means
that I'm going to condition my decision
about what to do next so which symbol to
omit next on at most the previous case
symbols and now this is an approximation
of the actual objective which is on an
unbounded sequence and unbounded prefix
but the the probability of the next
symbol given a bounded prefix and we're
reducing that to the estimating the
probability of the next symbol giving
the previous K if they exist so we can
again know we see the count based
methods show up all over the place we
can estimate this probability using
counts
so very trivially we can have a tabular
approach where we're going to collect
all k plus 1 grams so all all sets of k
plus 1 words that occur in our corpus so
that's kind of having a moving window
sliding over a corpus and recording the
contents of those window at each time
step and then to produce the probability
of the next word so the final word in
that window we're simply going to count
how many times we've collected the K
word prefix divided by the number of
times sorry how many times we've
collected the exact tokens seen in that
window divided by the number of times
we've seen the K word prefix so this is
pretty easy to derive by the definition
of conditional probability if you really
care to understand it in detail but I'll
leave that as an exercise so the problem
with this approach is that it's very
sensitive to the the order of our Markov
assumption if K is too small then the
language model is terrible so trivially
when you had everyone calls us like
iPhone poetry or something which is
frustrating to linguists to have this
idea since you know 50s but you know if
you have predictive text and you just
predict the next word you can generate
sort of vaguely sensible sentences of
English but you quickly notice when
you're when your language model has a
very low Markov assumption in that you
know the word the words being predicted
seem to depend really specifically on
the previous words so if you say I am
and then you'll see like a list of like
hungry lost on my way home those will be
very likely so suggestions given a to
order to markov assumptions but if
you're if you have something where it's
clear that you're talking about
don't use the bank example I shouldn't I
should come it's like if you if you have
a pre fit if you're starting a
conversation that says it's like you
came home at like 2 o'clock in the
morning last night therefore I am and
you don't want to say hungry at that
stage you want to say I'm angry or
annoyed because your partners woke you
up in the middle of the night
to predict annoyed it would be very
helpful to be able to take into account
more than just the previous two words to
like look at what's been happening at
the beginning of the sentence because
that's probably being a shift you know
your particular bias towards talking
about being annoyed or sad
rather than being hungry does that make
sense so a larger one obviously a larger
larger K a larger Markov assumption
allows you to condition on a longer
context a smaller K would probably
trivially yield a fairly limited
language model but when K grows and the
problem is that these particular counts
are going to become more and more
arguing to become a smaller because if
we had for example if you even if you
take you know millions of webpages a
10-gram so ten words a ten word sequence
is likely to be attested only once in
that even if you take five grams and
google has an Engram Explorer which you
can we actually collect counts from if
you want are probably going to happen a
few thousand times some six caroms of
seven grams only a few hundred times so
as you grow your order and Markov
assumption the accounts are going to
become poorer and poorer estimates of
the actual probability of that sequence
due to this sparsity issue so in a sense
larger larger K so we often talk about
an order and markup so I'm just going to
use K here cuz I used n in other places
but the larger your order your the order
of your Markov assumption the better
your language model should be but the
sparser your table is so therefore the
worse your estimation of the actual
probabilities becomes there are some
solutions to address this like smoothing
so can a certain a came up with this
idea in 1995 but overall account based
on Graham levels suffer from sparsity
even with more sophisticated and more
robust statistical methods nonetheless
until I guess really big Ellis TMS on
GPUs came along in 2014-2015 these were
still like the best language models we
could trade so sure take a 10-minute
break now and come back at three or four
past we've covered so far how we can
learn
of individual words and a lot of the
methods that we've investigated are in a
sense unsupervised and that they're
building entirely on an objective
defined over explaining the data as its
observed some text from a corpus in
order to build representations of those
words and then what do we use these for
well we're gonna see throughout this
lecture and also in the lectures that
you've seen in Oriels
lecture and I think that's nice Graves
is given a lecture yet ok next week you
might see also uses of word embeddings
there I don't know what exactly he'll
talk about but training your word
embedding matrix through these
approaches is an example of transfer
learning it's an example of saying okay
I have a lot of data that's not labeled
and I want to build good representations
for my words and then I'm gonna use
those as the initial state of my
embedding matrix in a translation model
or in the sequence annotation model or
an a sentiment analysis model and I
might fine-tune it by updating the
matrix while I go or it might freeze it
and say this is I'm transferring
knowledge from a larger corpus into this
specific task and because I just noticed
I don't mention this in detail later in
the slides let me just quickly point out
before diving into conditional language
models that this choice is actually
quite an important one so when we have a
lot of data and you're doing some
extrinsic tasks like translating or
recognizing textual entailment typically
what works best if you have enough data
is to just randomly initialize your
embedding matrix and learn it through
back prop alongside everything else in
your corpus but for some tasks where you
do not have enough data and enough data
really depends on the task and on the
signal you get from the annotation it's
found that fixing your word embedding so
not updating that during your model
training and updating every other part
of your model yields significantly
better results in that it's less likely
to overfit because those representations
contribute something a bit more global
from outside of your training data in
that task so especially like in question
answering when you may have heard of or
will be told about like the squad data
said if you work in question answering
my Stanford overfitting is a
a common problem there and I think
ubiquitin uniformly across the state of
the art models and a data set people
will pre train their word embeddings
with word Tyvek or glove which is
another approach and then freeze them so
it's a very it might seem almost
irrelevant given that you learn look
neural networks are in ten
differentiable so why not just learn my
word embeddings along with everything
else but if there's one area where
transfer learning has worked quite well
in natural language processing its word
embeddings okay so let's jump in so
we've seen how count based methods not
only could provide very naive ways of
producing semantic representations of
words but also of estimating the
likelihood of text we're not gonna move
on to seeing how neural networks can do
this and how not only can they do this
better but they allow for condition of
conditional language models because they
produce abstract representations of
features of any things any conditioning
information so therefore you can express
much more we can shape the condition so
you can shape the probability
distributions according into specific
tasks instead of just doing unsupervised
learning so before I jump into that let
me talk a bit about how neural networks
or a related approaches have been used
to do unconditional so normal language
modeling so it turns out you can
estimate the probability of a string
like you've like we do in a traditional
language much count based language
modeling by learning a function over the
probability of the next word given the
previous K word so with the same order K
Markov assumption using a neural network
or a log by linear model which is like a
neural network but without the
nonlinearities so the structure of these
is like that so that conditional
probability which is really the main
thing we want to learn from the data is
going to be estimating by for embedding
the token of each word in the history so
each word in our prefix we will simply
embed it and we'll project it according
to position specific matrix so the the
third word before our target word will
be projected using a different will be
embedded using the same embedding matrix
as the second word and is the first word
before but they'll go through an extra
linear projection which is dependent
their relative to position so that
capturing both the sort of context
independent semantics of the word
through the embedding matrix and the
context dependent dismantles of the word
through that position
positional matrix and usually you'll
have a positional bias it turns all the
the trend these days is to say that
biases don't matter so much because
especially if you're learning everything
in ten but it's nice to include them
somewhere so you you you embed each word
in the context you project them by
specific by position specific embeddings
you simply take the sum of the result
and then you put that through a function
Sigma which may be it which is either
non-linearity or something else and
they'll describe that you put it through
a projection back into the space of the
vocabulary so this is the loge it's
categorical distribution over what the
current word is and the softmax
operation turns that into account in the
parent primaries of a categorical
distribution which is exactly what we're
modeling here so the yeah the high-level
idea here is you simply embed the words
you project them into a single space
depending on their position and then you
predict what the next word should be if
Sigma here is a non-linearity like a
sigmoid or @nh then this is the neural
language models by Ben gol in 2003 which
I guess was one of the sort of
resurgence of like neural language
modeling before Ardennes and Ellis teams
sort of made to come back and if it's
the identity it's what's known as a log
by linear language model developed by
Andreini and geoff hinton in 2007 and
the fact that these are both roughly
equivalent is only something that has
been observed since then so it's not
like they just took an idea and made it
simpler right it they came at it from a
slightly different direction and the
sort of joint properties have been
explored since obviously if it's the log
of my linear models are advantageous in
that they don't contain nonlinearities
so if you're taking if you're doing the
cross entropy and log space you don't
need to actually take this off max
during training and you can be very
efficient and do this on a GPU super
super fast and if for some reason people
aren't super into
like my linear models anymore because we
can train deep l STM's pretty fast on
modern computing architectures but if
you really wanted to have a model that
was just good enough and super easy to
run on the say iPhone um that's
surprising no one's looked into
revisiting log by linear models
obviously the hip thing these days is
recurrent neural networks and you've had
a lecture on recurrent neural networks
already right so I'll just remind you
kind of how this how these work so we're
asked live by linear models and neural
module and neural language models by Ben
Jia from 2003 like remove all the issues
of sparsity from count based methods by
saying we're just going to use dense
vectors and learn the features
independently so just in how we were
removing sparsity and feature
engineering from word vectors by using
neural embedding methods those are doing
they're removing the feature engineering
and diversity issues from the language
modeling side of things through the same
method but they still rely on this order
K Markov assumption and that you're
still just doing this projecting the
previous K words into the same space and
you're ignoring the prefix before so
this isn't exactly modeling the total
probability of the data recurrent neural
networks allow us to relax this
assumption by not specifying K
explicitly so technically you've seen
neural networks have a basic construct
this recurrent cell which at each time
step tape takes a projected in version
of the input so this would be the input
word and you embed it and you put it
through linear projection and then you
add it to your hidden state perhaps with
some gating if you're using those TM and
then condition on that input and your
previous hidden state if you're doing
something like a GRU or an LS TM there's
some complex gating architecture in here
that updates the internal state and that
gives you your next state and based on
that next state you also are going to
project back into the vocabulary and
predict what the next word should be for
example so the activations at each time
step of this recurrent cell which we
unfold over the sequence to produce a
sort of variable size network with
repeated parameters model the
information of the entire prefix of all
the words that happen
or in theory in practice there's still
there's still there's still modeling
some bounded history so there are only
going to really remember information I
mean so trivially this is a fixed size
information reservoir even if you're
using continuous values they're going to
be you know discretized at some point in
your computer so there's no possible way
that this can learn to just like capture
completely unbounded information from
history and you'd get something out of
it right at some point it will get lost
in the noise so in practice what we
observe for our n ends is that they're
going to capture dependencies that are
entirely based on the data and the
objective so if they need to you know
never really look at things that are
more than five or six words but back in
to be a good language model then they're
not really going to remember that
information in a useful way
and people have like tried to visualize
this by looking at sort of the gradients
of those dependencies as you make them
longer and longer in language models but
that's really kind of like meta analysis
so in practice the bounds still exists
but unlike an order an or order K Markov
assumption you don't need to provide it
the data provides it and when you're
training the model and it's on function
of your data of the size of your model
of the complexity of your dating
architecture and in the sense that's
really the main thing that
differentiates an LS TM from a simpler
RNN is that they make it easier during
inference during training to capture
longer range dependencies but they're
similarly expressive so you can do
language modeling with an RNN and you
can condition through the previous state
on the history of the sequence the
unbounded history of the sequence but
because we're doing everything in
vectors right and these are the sort of
nameless and abstract features these
vectors can actually represent
information that is beyond the boundary
of the sequence that just if information
from another part of your data right so
you could imagine that you have some
random information beta and this could
be another sentence it could be a
representation of the topic of the
document it could be the sentence you
want to translate I mean this is
basically the basis of sequence the
sequence
as we'll see in a second and produce an
embedding of that and just say this is
the start state of my language model and
now I'm producing my conditional
probability that factorizes is actually
expressing a global probability over
this string giving the skanda
conditioning information beta and you
could as I said condition by setting
that as the start state or you could
feed it as an additional input the nice
thing about neural networks is that
these you can be really flexible with
this kind of like network topology and
explore different variants on the models
without really needing to think about it
in too much detail and the model will
just like work around it as long as
everything's end-to-end differentiable
so this idea of conditioning on some
information to train a conditional
language model has yielded a variety of
really interesting applications of this
kind of architecture or set of model of
set of architectures by using
conditional language models to do
transduction and it turns out a lot of
problems in natural language processing
but also other areas are castable as
transduction problems is transforming
one sequence or several sequences or
other modalities such as images of data
into another kind of modality so
transducing French into English
transducing code into the results
transducing and image into captions
translation parsing computation all
these things are sort of transduction
and therefore something that you can
express using this fairly general
framework of conditional language
modeling so it's really cool we've gone
from something which is just okay here's
the basis for unsupervised learning as
adapted to text to a very simple kind of
architecture that you can use for all
sorts of actual tests you might care
about if you think language modeling
itself is boring which most sensible
people do so again you've probably seen
sequence to sequence mapping in our
girls lecture I'll quickly revisit it in
my own sort of formalization and then
we'll get onto some application specific
to language are there any questions
before I jump into this
okay so transit so transduction or
sequence to sequence mapping is the goal
of transforming some source sequence of
information so some symbols from a
source domain into a target sequence
and typically how this will work is
we're going to construct a
representation for our source sequence
or conditioning information s and we're
going to model the conditional
probability of the next target word in
the target sequence given the target
words you've generating generated thus
far so this allows us to specify a very
simple informal algorithm where you read
in your source sequence or perhaps read
in the source image or however you want
to encode that source information to
produce the representation of the source
and then you train a model simply to
maximize the likelihood of your target
sequence condition on that information
that test time yo J you'll once again
encode whatever conditioning information
you have like a new image or a new
sentence of French and then you can
generate the target sequence using
greedy methods beam search or a variety
of search processes guided by that
conditional probability distribution
you're learning so the obvious were in a
very famous approach although it was by
no means the first approach to sequence
the sequence mapping using neural
networks is in this paper by szyskii ver
and Co in 2014 so here we're going to be
learning a conditional model over
English given some French conditioning
sentences which is effectively what
translation is in a very simple way how
this works is you'll have your source
sequence of French tokens and you'll
embed the individual words and feed them
into recurrent neural network and then
just take that network forward so
condition it can updating the
representations based on the prefix
you've read so far and ignoring the
output because it doesn't matter during
the encoding phase all we care about is
the final representation of that
conditioning information and then you
can actually use the same network if you
have for example an explicit separator
symbol or a different network with a
different topology etc so start decoding
so condition on the representation
you've had at the end of your encoder
you can start outputting the words of
the translation which are during
training observed and during tests are
going to be sampled and typically you
will take the sampled word and feed it
back in as input a lot of regressively
during the test time
and during training time you'll feed in
always the correct word regardless of
how the distribution looks in order to
condition the information on on they
sort of like a stochastic choice that
was made in the output so simply by
maximizing the likelihood of the target
words conditioned on the source
information we have a perfectly
functional translation model which works
impressively well given how simple it is
compared to a lot of the machine
translation that existed before so there
are some issues with this very very
simple model though that we're sort of I
guess glossed over when it was initially
presented but a lot of the recent
developments and the recent I mean over
the last three years in or four years
now on this in neural machine
translation have come out of trying to
overcome this problem is that if you
just treat the sequence 'men if you just
do sequence the sequence modeling like I
described you you have this bottleneck
between the source and the target which
means all the information from the
source needs to be contained in the
activations of your neural network at
the boundary between the two sentences
so this is a problem of non adaptive
capacity so for the same reason that a
it's unreasonable to expect the
activations of an RNN to model a
completely unbounded prefix here it's
unreasonable for them to be able to
compress because that's effectively what
this is the information from source
sequences of unbounded lengths right so
the longer and the longer your sentences
are the more your fixed size information
reservoir is going to fill up to the
point where you assume that there's
going to be some compressive loss as it
tries to decompress it into the target
language we also this also causes
prompts during parameter inference so we
find that the target sequence modeling
in this architecture is dominated sorry
the the modeling of the target sequence
language dominates training what this
means is if you're training a sequence
the sequence model without attention
like just with this kind of structure
and at random points during training you
say okay let me test it let me just feed
in a sentence of French you'll see that
doing maybe the first like 80 or 90
percent of your training time
the sentences that will come out will be
English will be correct fluent English
that will happen actually fairly early
on in training you will start producing
things that look like English but that
English translation won't have any
relation to the French text so it's
learning a marginal language model
before it starts to actually learn the
conditional model and the last bit of
training and usually quite far on in is
learning to propagate the dependence
that learn the dependencies between the
source and the target and this is simply
because you can observe here what you
what the model observes what the where
the stochastic choices are sort of
realized is only on the target side all
right so the only point you get error in
the cross entropy between the
distribution over what word belongs here
and what word belongs here is when
you're decoding which means that the
parameters of this whole section of the
network and the embeddings of the source
words and the dependencies between this
position and the end all have to go
through a bunch of nonlinearities which
we know typically squat for cause
banishing or exploding grading issues so
the noise the the the gradients can
become more and more unstable or
typically reduced norm for most Morin
ends as it goes back through time such
that you're getting very good updates
here and learning in the language in the
target marginal language model and only
because eventually the gradients if ik
is sufficiently sort of noiseless where
the expectation is sufficiently close to
what you're actually trying to adjust
the model to does the encoder learn to
act appropriate the in condition the
model so in the sense this fixed size
assumption this bottleneck you have
between source and target is one of the
main reasons that it's almost amazing
that this worked in the first place as
work despite its archaic architecture so
you have covered again started keep on
asking this I should have just looked at
the slides you've covered a tension in
some form no okay you've not seen
intention wonderful okay
so attention is one very powerful
solution to this that's inspired by
traditional machine translation
approaches where they're a separate tool
typically would provide an alignment
matrix so when you have a serve source
and target
strings you could have a set of
probabilities over which words and your
source target aligned to which words in
your target and a lot of the traditional
statistical machine translation models
like IBM's model 3 on words or 2 on
words and incorporate that information
so alignments a way a soft Layton Way of
conditioning on this sort of alignments
that exists between target words and
source words and for some sentence for
some language pairs these alignments are
almost one-to-one if you're translating
you know English into German or French
into English and languages that are very
sort of closely related within the
indo-european family and have a sort of
a recent common ancestor relative to for
example Mandarin and in English it's
almost like if you translate the words
individually and roughly reorder them
you're good and that's what models
effectively are doing so attention works
like this you have a assume that you
have an array of vectors produced by the
encoder that represents some data and
obviously we have that in this case when
we are encoding we're only using this
red vector to decode right that's the
state that matters but you need to save
the state of all the previous vectors in
memory because you're gonna back
propagate and you're going to need the
state of those part of those pre of
those activations in order to properly
calculate gradients so you've it you
have all this information sitting around
in memory and attention is just about
using that assume you have that matrix
that represents some data and also the
position matters so just like here you
have a you have a set of vectors and
this vector these two vectors here
corresponds to this word here so the
position isn't is important your
controller is going to so the decoder
cell is now think of it as a controller
and that's a controller that's dealing
with in for in with input and output
logic in your model right in the very
trivial case of just dealing with okay
here's the last word I translated and
here's what I was doing before here's my
internal state so I'm gonna predict what
word to do next but now it has access to
an extra component which is this array
of vector which is a like a read-only
memory if you will and you want to read
that data array at each
step using the controller in order to
make better predictions about what
you're doing and then turn this means
because you're reading the memory and if
you do this to the differential
operation you can accumulate gradients
in your memory and then use those as
well so if you really want to think
about this is like a 1/8 as an API isn't
removing ourselves from the specific way
that attention might be done in this or
that paper or what controller you use
whether you use an LS TM or an RNA or
GRU attention kind of broadly has two
forms and note that both of these forms
which I'll show you the second one of in
a second are themselves kind of are n
ends right they have inputs of outputs
so everything inside this box is like a
very structured recurrent cell and they
have previous state in next states and
they're going to be updating those so
early fusion is going to be the approach
where at each time step you you observe
an input so the let's say the previous
word that you just translated in your
English sentence and based on your
previous state so on the previous state
of your controller you're going to read
something from the memory and combine it
with your input you'll update the
controller state which you'll then pass
back with the current state of the
memory is the next day of the overall
recurrence and you'll make a prediction
on the output so early fusion is
basically I look at what III look at
what I've just translated I look at what
I have left to translate I update my
sort of internal state and I predict
what the next word should be i will not
go if you read through the mathematics
here but it's here for your if you want
to look at roughly a formalization of it
in your own time late fusion is a very
very closely related approach so in this
case I read an input into my controller
and based on my previous the previous
state of my controller I update my
internal state so I look at the I look
at the word I just translated and I
update my internal state based on what I
think I have left the translate in my
head and now I've updated in my state I
look at the memory and directly pick the
word that's in the next word to
translate and then based on my current
state
and that selection I pick an output and
so that's basically what this is these
are almost trivial differences except
early fusion has you're doing you're
attending before non-linearity and late
fusion after the non-linearity so they
have pros and cons in terms of experts
if ax T in gradients but they're both
very similar when it comes to
translation so equally these are
formalized here and the concept of
attention do people want me to talk
through the actual operation of
attention I don't want to rush over this
raise your hand if you actually want to
get the dirty details right now okay
someone to raise their hand stuff like
everyone else okay
let's let's let's do let's do late
detention because that's easier so my
previous state is as two components here
it has the previous state a little h t
minus one of my controller so that's the
previous state of your RN n and it has
my memory so my array of vectors i have
from the encoder and so I don't put a
temporal index on those because these
are going to be unchanged once the
encoder has been run that is the same
during every step of decoding I'm trying
to remember what exactly I wrote here so
I'm going to this looks like an N but it
should be an H I'll fix that so I'm
going to sorry yes I remember I wrote so
I using my using my controller update to
logic so this will simply be an LS TM
update or an RNN update depending on the
nature of my controller based on the
currently read symbol which is the
previously translated symbol and the RN
ends previous state I will update my
controller logic and have a vector which
is kind of the intent of what I want to
translate then I will pack that can that
updated internal state along with the
state of my memory as the next state for
the recurrence and here's where the
attention comes in I'm going to have
based on the current state of my RN N
and my attention matrix and again to
apply a differentiable function which
will produce a vector and so how this
typically
and I'll just sketch it out verbally
because there are so many variants is
assume I have some way given my current
vector which I will treat as a query so
this imagine this is my query vector and
I want to retrieve something from the
memory using it I'm gonna see how
similar that query vector or some
transformation thereof is to each
position in my attention matrix so
that's gonna give me a single single
number for example I'll take an inner
product and I want to turn that set of
numbers into distribution over positions
of my attention matrix so what the easy
thing you can do is apply a soft max
operation so the numbers I've computed
through that similarity kernel becomes
just the low just to my soft max so then
I have a distribution over positions and
now if this was an actual lane variable
model what I could do is sample a vector
according to that distribution and say
that's my that's the return from my
query right that's what my memory has
returned but that while we do have
inference methods to deal with that like
reinforce or you could try and
approximate the marginalization of that
decision instead we've postulate a mean
field approximation where we're simply
going to take the weighted sum of all
the vectors in my memory weighted by how
much it's supposed to be returned so
that's basically taking the expectation
of the vector according to that
distribution so that vector is what will
be returned by this section I'll combine
it with this according to a composition
function like an MLP or whatever and
then just predict the next word so in a
sense this the logic here is is fairly
simple and if it takes a while for you
after there like stare at it until you
can convince yourself that it's
basically just like here's some
information give me a vector back and
then I use that to predict its own it's
just peeking at the memory right that's
just the nice T of this because of these
operations like attending and then
combining are typically linear
operations so softmax
and inner products or MLPs
the whole thing is and then
differentiable so that you're passing
gradient throughout your system when
you're doing this so I'll talk about
about some applications of sequence to
sequence and attention and then LP now
to make this a bit more concrete
the first is skipping this bottleneck so
as I said this bottleneck is annoying
because we have all this redundant
computation that happens before the
vector
red but we only condition on the vector
in red and that causes a bunch of nasty
issues so my can if instead at each
stage here before predicting the next
word I got to look at what happened
before which is my attention matrix
using the logic of my Oran and at this
state I could capture longer-range
dependencies instead of doing it through
my own internal dependency tracking
mechanisms with MV R and n based on the
representations that were produced
earlier which is effectively what we
want so attention does look a little
like that is that at each stage when I'm
about to make a prediction whether I use
early or late attention I'm conditioning
on all the information before and
filtering out so those I think are
salient based on my current
representation because that operation is
differentiable the gradient not only
flows through the computation that's
happening in the R and M so those are
they the paths where you're expecting
the gradient to be attenuated but also
through these green lines here the at
each stage where I'm comparing against
all the previous vectors and my atention
matrix in order to decide which word to
translate because that's the
differential operation I get gradient
that immediately flows back to let's say
the first word in the sentence and
completely skips the bottleneck so by
having a memory that you can attend to
that you can look back into you're not
only solving a forward problem that you
can capture long range dependencies more
easily but training becomes easier
you're passing you have a channel where
gradient can pass without as much noise
back into the encoder and help you train
the encoder faster which
surprise-surprise allows you to train
your overall translation system
significantly faster as well another
cool application of this and not just of
attention but of sequence modeling is
textual entailment so this is moving a
bit away from the language modeling
objective but using the exact same
architecture and this is really one of
the strengths of neural sequence
modeling is that the same architectures
that work for translation might require
complete retraining but they're
typically a good starting point to do
completely different tasks like parsing
or textual entailment which in
traditional approaches might have
required a very different approach built
on like I'm going to use the the parse
information differently and come up with
a new algorithm etc so texts don't
almond is a pretty cool task and it's
structured like the
you have two components one is a premise
so that's a sentence that you're told is
true and then you have a hypothesis and
what you need to do is classify the
relation of the hypothesis of the
premise so here I'll give you some
examples I have a premise here in three
hypotheses that you have to separately
classify the premise is a wedding party
is taking pictures and a first
hypothesis is there is a funeral so this
sentence obviously isn't really
consistent with there being a wedding
party taking pictures so I could say
that those that statement contradicted
by the hypothesis or vice versa there's
a second sentence which is they are
outside so this is where it gets a
little more fine-grained is that that
sentence is actually quite plausible for
a wedding party right if you like if you
really let your prior knowledge of
wedding parties or in fact that
knowledge that's in the data inform your
model you might be like okay that's
really plausible but from a logical
perspective that's neutral it's it could
either be the case where it could not be
the case it's not like directly entailed
by the by the premise and then finally
we have the hypothesis someone got
married which is trivially entailed by
the premise and in this case requires a
little bit of like external knowledge
like for example the relations between
weddings and people getting married you
have another one a man is crowd surfing
at a concert that's obviously if the
hypothesis is the man is at a football
game there's a contradiction unless it's
a really really specific kind of concert
I guess the Super Bowl right okay yeah
the man is drunk this is really
plausible but again that's like we're
talking about logical entailment here so
plausibility is something that you
almost have to resist with in your model
and so you models that exploit just raw
statistics might get this wrong and then
the manners of the concert is trivially
trivially follows from them so it's a
scent you understand this you get two
sentences and you need to classify into
one into two or three categories
sometimes some versions of this task
tell you like differentiate entailment
from ro-ro's entailment which is the
case where the hypothesis entails the
premise or the premise entails the
hypothesis and you need to distinguish
this but in this wonderful data said by
Stan Bauman from 2007 15 SN Li you only
had these three classes so there are
variety of approaches you can do here
involve attention and that look a lot
like translation models but aren't
translation models so my internet in
Oaxaca show a few years ago worked on
these models you'd simply take an LS TM
and instead of translating the premise
into the hypothesis you simply just
encode both of them except while we're
reading the hypothesis you get to attend
back to the premise so this seems a
little different than what we're doing
in machine translation although if you
look at the model topology and you
squint at it it's almost exactly the
same thing as translation except we're
only predicting that class based on the
final state but while we're reading
we're using attention for a different
purpose and this is nice because because
we have attention we can instead of
having the model having to decide and
Talman for contradiction just through
the dynamics of the RNN it gets to look
for things that it thinks indicates
contradiction or entailment almost
component-wise so for example we and we
can visualize these as heat maps so here
if we have as a premise or you have to
sort of like tilt I'll read it out for
you
the premise is a young girl kisses a
ring bearing boy at a wedding and the
hypothesis is two kids are playing tag
and so here it's these are strong
indicators for entailment the the kids
kids mapping two kisses and boy sorry
the girl and boy so I can split those
dependencies it also sees perhaps as an
indicator against entailment that
they're playing tag yes I think it was
probably rejecting it so you can you can
you can try and understand what the
model is basing its decision on and
obviously the presence of this
qualitative quantitative lis helps the
model distinguish entailment from rome
contradiction from rejection but and
while this looks like an alignment
matrix well it looks like these are
worth the words correspondences remember
that these models when it's doing
attention and you can try bi-directional
honesty and variance here two are also
conditioning on all the words around
them or at least in the prefix so it
there's already some disambiguation
baked in right it's like when it sees
girl and it compares it to kids it's
like it sees girl conditioned on a young
and kids conditioned on two so I they
have this like richness
you don't have in simple sort of models
based on the parses again sequence of
sequence models using the exact same
architectures we've used for translation
have been used for a variety of other
approaches like large-scale supervised
reading comprehension so testing and
training or training models to answer
questions conditioned on documents and
therefore being able to you know read
and arguably understand some aspect of
the document because they can answer
questions about them so how this works
is imagine that you had the task is
you're gonna see a document you're gonna
see a question and you're going to need
to produce an answer this if you really
want to look at it under the view of
translation is a translation problem
you're translating the document and the
question as a source into your target
which is the answer the questions are
typically in this formulation of the
task close form sort of fill in the gaps
but other datasets of other sort of
variants on this in here - attention as
you know it from from textual and Kalman
and translation can be adapted quite
well so you can read a document with a
big LS TM you can read your question
with a big LS TM and while you're
reading the question you can attend over
the documents several times to try and
see it's like okay maybe maybe this is
the answer maybe this is the answer and
then at the end decide what you think
the answer is which again allows us to
produce really nice heat maps so
attention now can tell you a little bit
and obviously there's other parts of the
model which aren't visualized in this
but we can somewhat peek under the hood
to see when it's trying to answer a
particular entity and identify us to see
Seiler as X it attends to several
possible candidates and only through
reading the question iteratively we'll
refine well like Centre on the correct
answer we've anonymized the entities
within the stories not because of
privacy concerns but because this
prevents very drastic overfitting and
this is a fairly common technique in
these kind of them these kind of tasks
any questions on that before I move to
the final section is just all subdued or
is
really all very sensible and very well
explained to just nod okay if you have
any questions after you can come to me
at the end of the lecture so as a final
sort of like stop on our what metaphor
am I going for here on our train ride to
understanding language within your own
networks I want to talk finished by
talking about about composition so we've
talked about like words we've talked
about tasks than involve transforming
words and underneath those tasks there's
typically the need to be able to
condition on larger units of text that
we embedded with Ellis teams but there
are obviously other ways to produce
representations of like the source text
or of an image or of conditioning
information and that sort of falls under
the sort of topic in natural language
understanding of compositionality or in
composition so how do parts like words
or characters combine into holes which
are more than the sum of the parts so
obviously a sentence has made out four
words but it's not you know the order
matters the presence of other words
disambiguates so composition plays a
very rich role in how we understand
cognitively sentence ISM so how can we
model this with neural networks or other
methods and I'll talk about this a bit
from the under the view of
classification since we've focused so
far on the transduction or sequence the
sequence modeling although we did see a
few classification tasks so a textual
classification involves the prediction
of a label such as you know it's like
positive or negative or a scalar value
which is how how likely for example a
tweet is to be coming from a bot versus
from a human before and and these these
these labels or scalar values are
predicted from a sentence or a set of
sentences or a set of documents
depending on your task and so typically
or you want to do is you want to produce
a representation of your documents or
your sentences or your phrases and then
use the to jointly learn represent at a
specific representation and a classifier
from that representation to your label
so that you can solve your task yeah
that's more or less what I said in much
more words
so how can we produce a representation
of a sentence if you've have
representations of words which you're
either jointly updating or that you
obtained from a word embedding method
such as those that we've explored at the
beginning of this lecture well there's a
variety of algebraic operations that you
could do before you jump straight into
neural networks the most trivial one is
you could just take the sum of all the
vectors or their mean you could take the
product of all the vectors component
wise or if you think okay I want
something really simple an algebraic but
not relying on commutative operations so
I've trivially the first two options
will destroy a word order so you're
guaranteed to have the exact same
sentence representation for Man Bites
dog and dog bites men and that might
lead some confusion for your classifier
if you encounter that sort of sentence
you can do things like this which is for
example will sum across all the pairs of
words in your sentences and apply Tammy
chess or sigmoids or sort of symmetry
breaking operations that will make the
whole thing not not entirely commutative
so these are all very very sort of
simple operations we understand
algebraically they're fast and they
don't add parameters I mean the only
parameters are your embedding matrix so
here the representation is that you
learn under the constraint of the
combination operation do all the work
and you might think okay this is just
like weird preliminary to the more
sophisticated composition operations
that adds about two present and you'd be
partially right
but it turns out that's the this is a
very sensible first thing to try I mean
just like bag like aggregate all your
vectors with a sum or a product or a
mean and then just have a linear
projection into your labels that should
be the first thing you implement because
a will take like three minutes and be it
might work and if it works really well
then you know that either your task is
very simple and you know how to
interpret the gains you get with a model
that took you three weeks to implement
or it might be if you're in a business
and this is like sufficient results for
your application then why would you do
something that's going to require GPU
and you have something that takes like a
microsecond to compute so always try
really simple composition methods for
because they frequently work very well
and you'll see frequently in the
literature people referring to like bag
of vector methods where they some more
mean the vectors in a word as a sensible
baseline so do you implement that as a
baseline so as a connelly this is not
very flexible and that you can't
incorporate some tactic information it's
dubious that it's certainly not going to
capture word order
it's dubious that it's going to capture
any sort of disambiguating effect
although maybe the product will allow
you to say things with a low norm are
going to become even lower and that will
that will work but and it puts a large
you're putting all the sort of work from
your model into the representations so
that is going to be limited in terms of
the expressivity of your model so the
solution could be do what we did in
sequence the sequence modeling just run
an R and N over your sentence and use
the final state or use a bidirectional
oil RNN or use a bi-directional STM or
use a neural Turing machine or whatever
you want like a recurring the really
structured recurrent model could
probably produce a very good
representation of the sentence and
that's a perfectly sensible thing to do
but what other options are there that
might use information that we might have
handy and so the sort of two kind of
models I'll get at today in the
remaining part of this lecture before
concluding Center around the use of
trees and these people natural language
processing people especially with the
advent of LST ends and deep learning
methods have had a bit of an on-and-off
relationship with trees where initially
people were like okay let's use trees to
guide neural networks and then everyone
was like ok well it's like we can
translate with LST m so syntax is dead
and we don't need to use trees anymore
and now everyone's like ok maybe we
should be using trees again it's like
it's like it turns out it's a very good
inductive bias and if you're doing
instant pick maybe for language this is
difficult because we don't know the true
grammar of of English or other languages
and we usually have to train and model
in that moment but imagine you're
running a neural network that needs to
produce a representation of logical
expressions or code or maths it needs to
generate code for all these formal
domains we
no the exact syntax and the relation
between syntax and semantics so why not
exploit it I mean it would be crazy to
say we have this information and but the
ls team doesn't need it so let's just
use the Ellis team so trees trees are
back and I'll give you a quick snapshot
of two ways in which you can use trees
but before let me know as an
introduction in case you really have
never taken the class and linguists in
linguistics many linguists believe that
language is tree structured and by that
we mean that underneath the surface form
of an expression like the old cat 8 I
didn't pick the sentence that's I think
sanded a weird sentence underneath the
surface form of this string there's a
latent tree or perhaps several trees
that guide our interpretation or
understanding of this sentence so each
sentence has Alain parse tree or
distribution over trees and the tree
tells us syntactic information sort of
like that does a determiner olds an
adjective cats a noun it gives us
combinatorial information that for
example old applies to cat because both
combine to form a noun phrase or proto
down phrase that the noun phrase is only
typed correctly or valid if there's a
determiner in this case and that that
noun phrase is the subject of eight
because it combines to form a sentence
so syntax isn't just about like type
checking your natural language it's also
about trying to understand the
relational information expressed in the
sentence and obviously there exist
parsers and there's a large body of
literature from you know decades and
decades of natural language processing
in linguistics but there exists
statistically driven tools that are very
accurate for languages like English and
not so accurate for other languages that
give us a distribution over possible
parse trees given sentences that allow
us to get trees on the fly for sentences
in your training set yes
that's a great that's a great question
ascends get to that in about two slides
but yeah there are they're having to
work on neural networks augmenting
statistical parsers and you could
imagine formulating it completely and
then model and I guess we'll see a
variant of that in a minute where you're
jointly learning the pars are in the
actual composition function but let's
start for the case let's start by
assuming that we do have parsers that
are we don't care how they're built but
they give us like a tree for sentences
and now we the question we want to
answer is how do we use this information
this added information in addition to
the words and my symptoms and their
order how do I use this structure in
order to guide composition so the very
basic idea behind tree are aren't ends
so they are recurrent sorry the
recursive rather than recurrent neural
networks but trivially an RNN is an a
recurrent neural network is also tree R
and n with left branching or right
branching I can't remember which but
it's you know you combine combine
combine and the trees always the same
but the basic idea is you're gonna use a
sample of the maximum likely parse tree
that your parser gives you to guide
composition by applying a function to
produce a representation of a parent in
the tree given as children so for
example in this case we would start by
producing embeddings for the old and cat
and eight and let's take the simplest
case where there's just one composition
function and our parse tree is
conveniently binarized which means
there's always two children which
conveniently is the case of my example
but obviously you need to figure it out
how to deal with the case where it's not
binarized so the vector for old and cat
and i have a my composition function
which is going to let's say concatenate
them and project them through an MLP to
produce a single vector for old cat
which is in the same size or space
because it's obviously gonna be
recursive and I have a vector for the
and my vector for old and cat I apply
that same function get a vector for the
old cat and do this again for the old
cat and eight to get the vector for my
sentence so this is just like structure
very much like an RNN recurrent neural
network is taking a repeated unit of
computation and unrolling it across the
sentence to dynamically
do some neural network based on the
length of your sentence a recursive
neural network or tree yarn in which
there are no I'll call tree Ardennes to
avoid confusion because RNN is taken
uses the syntax of the tree to
dynamically produce the topology of your
network with repeating units of
computation now because we not only know
the topology of the tree but usually we
have type information so for example in
this case we know that there's a
different sort of combination happening
when I combine old and cat with the old
cat you can incorporate that by having
the parent via function not just of its
children but also of the type of the
operation now because there's usually in
a grammar a limited number of these
types of these operations the way you
typically implement this is just have a
different composition function dependent
on what the actual syntactic rule used
to combine was does that make sense so
here we assume that we have the tree
right and then you combine if you have a
distribution over trees and then you
could take like the top K or you could
like do a Monte Carlo estimate and
marginalize but people typically assume
that they just sample the most likely
tree or just greedily take it took the
most likely tree from the parser so I
mean here we've seen a very generic way
of describing the composition but this
applies to pretty much every variant in
this model where this could be something
like an MLP like I suggested or in there
have been more sophisticated composition
methods that mimic what we do with LS TM
so we're in gru so where you have a cell
and an actual activated State and you do
some gating so there's tree l STM's that
are effectively the tree analog to l
STM's and rnns and those work very well
in that they propagate gradient down
nicely but even without the tree LS TM
compared to an Oran n where the gradient
has to go linear like the distance the
gradient has to go to capture dependency
is linear in the length of the sentence
here is actually assuming a sort of
vaguely balanced tree log linear and
that's that's nice okay so
did I just describe this oh yes so we
have a variety of corpora where this
which we can use to to Train
tree-structured models so obviously you
could just run a parser over any data if
you have labelled data and then use this
kind of method but parsers aren't
exactly perfectly accurate so there's
been a lot of work in like paying
linguistics students in Stanford I'm
assuming an honest wage to annotate
corpora where we had objective labels of
what the sentiment of an expression was
and that's great because you can you can
you have high quality annotations there
and you can even even have like a
variant on their corpus where there's
sentiment labels on all the sub phrases
or the branches so you can like do full
supervision and all that but technically
you don't need that sort of stuff
anymore and obviously as I said you can
use parsers to annotate other text
classification and because that's its
training data you can do that and ask
your linguists to correct the parsers
because you still do need very clean
data the problem is you don't have
unless you're really rich you don't have
the luxury of having a linguist on hand
at test time so it's like you've trained
your model on tree data and then you got
to do this on tweets and run it in the
wild and people write all sorts of
different ways on Twitter that are going
to probably break most statistical
partners so and even if you don't even
if it's not Twitter and even if it's not
sort of like slang you're unlikely to
have very very very high-quality parses
for test data and you don't have a
linguist to fix it so it can be quite
difficult to actually run these at test
time unless you have an idealized sort
of test set so the solution well one
solution to this is to say why don't we
jointly train composition operations
with a parson so this is what Sam
bellman from anyway you did two years
back so exploiting the fact that there's
an eye there's an isomorphism so there's
a structural identity between trees and
a context free grammar and sequences of
shifts and reduces
so this is called shift reduce parsing
and I'll briefly give you an example so
I like your if you're if you're reading
if you're reading through the sequence
of the cat sat down shift means that
you're moving the symptoms you're moving
things on to a buffer and reduce means
that you're doing a combination
operation so the branching structure of
the tree actually directly and uniquely
translates into two times t minus one
the shift reduce operations for keywords
and the sentence actually I'll give you
an example in a second so we have the
cat sat down and then imagine that you
know that these shift reduce operations
because you knew the tree let's run
through the actual running of the shift
reduce parser shift is going to you have
a stack which is your sort of working
memory and you have a buffer which is
all the words that you have left to
process so you're going to start by
shifting the first word onto the stack
and then the shift reduce parser tells
you that you're going to shift the
second word onto the stack and then
reduce says that you're going to combine
the top two words on the stack in this
case to apply a determiner to a noun and
form a noun phrase then you're going to
chef sat onto the stack and then shift
down onto the stack and then reduce is
going to combine the top two words sat
down in order to form a verb phrase and
then finally another reduce will form
the sentence from the noun phrase and
the verb phrase so this is just building
another way of building the tree by
linearizing linearizing the operations
so it turns out if you imagine now
instead of words these are the vectors
for those words shifting is still going
to be the same thing you're just moving
a vector from a buffer to stack and we
can perfectly well do that and a
reducing is going to be applying a
composition function so your tree are in
an update or your tree lsdm update and
producing the compose putting the
compose vector back on the stack then we
can see how this is going to be jointly
you can you can if you have some way of
predicting the shift and reduce
operations you can jointly parse and
compose and then try and train both
based on the representation you have at
the end here so the high-level idea spin
is about doing just that you're doing a
training time you know you still have to
use a training time so you know the
shift reduce ground truth and
conditioned on the
vectors at the top of the stack and at
the top of the buffer independent of how
many there are below those you're going
to decide you're gonna have a neural
network that decides shift or reduce so
that's binary classification but we can
train that because we observe the result
of that classification during training
and you can just take Bernoulli
cross-entropy and if we shift remove the
word embedding and if we reduce we
compose and the composition function is
also differentiable so because we know
what the sentiment that the end of the
result is we know what the label the end
of the classification is so if we
predict the label based on this
representation then the cross entropy of
the correct label compared to the
distribution will back propagate through
all these composition operation shift is
the identity and then therefore we have
gradient on every part of this operation
with regard to the loss so we're
learning jointly a sentence
representation from the classification
loss and shift reduce from the parse and
both are jointly conditioned on the same
word representation so it's a powerful
model that allows you to deal with trees
when you don't have them testing and
really quickly I'm going to talk about
the high level sketch behind
reinforcement learning spin so this is a
variant that Denio Gotama possibly two
years ago so what if you don't want to
have trees during training time and this
is the case if you don't you know have
annotators to annotate your data set so
you don't have those shift and reduce
operations or you might believe that
linguists are wrong about the tree
structure of your language and that
that's providing unnecessary bias all
you want in your model is to say that
there is some tree structure underlying
the expressions but you want to jointly
infer that structure and also they
actually learn to compose and predict so
we can take spin the exact same model
and the only thing we're assuming we
don't have is the training labels for
shift and reduce so now you're making a
latent discrete discret the decision
during training we have to actually
sample from that distribution rather
than take the actual true label but
that's the only component that's non
differentiable in our model so you can
use reinforce so a reinforcement
learning style objective to estimate the
gradients unbiasedly for that lane
variable while everything else stays the
same
and what's the reward in this case if
you want to think about it in terms of
reinforcement learning well is simply
going to be the cross-entropy of the
correct the negative cross entropy and
the correct label so the same objective
we're minimizing through backdrop for
the rest of the model the negated so
whatever we're maximizing has to be
minimized by the whatever we're
minimizing the loss were minimizing is
going to be maximized by the reward
model because when you're minimizing the
loss you're maximizing the reward and so
this jointly learns to produce parse
trees and export them during composition
unbiased by what linguists might think
the trees should be and it's interesting
that it does produce some trees that
resemble what linguist suggests but
sometimes it produced trees that don't
fit on linguistic intuitions but are
actually significantly better for a
particular task so they might not be
reusable so it's just a really quickly
wrap up because I have like a minute
left language is a super difficult topic
and we've only scratched the surface of
some of the very simple issues that I
could cover in the two-hour lecture as I
said at the beginning there are many
natural language processing courses and
that go into like the bleeding edge of
research including lectures by me and
some of these that are I think actually
fascinating because they try and tackle
sort of the very definition of
intelligence and how we assess them so I
really recommend looking into this and
also if you're interested in this field
know that it is really new that there
are so many open problems some of which
you've actually brought up in questions
during this lecture so there's many
opportunities for progress which is
always a good thing to hear if you're
interested in continuing doing research
on a topic so I encourage you to look
into it if you if your appetite has been
wet by this lecture and explore the
detailed and open arena that is NLP with
deep learning thank you
[Applause]