Robust Learning via Robust Optimization (Stefanie Jegelka, Foundations of Safe Learning Workshop)

all right let's let's continue welcome
back our next speaker is Stephanie
yogyakarta Stephanie is a X Windows
concert team career weapon professor at
MIT she got her PhD from ETH Zurich and
the Max Planck Institute for
intelligence system at the two places
her research interests span the theory
and practice of algorithmic machinery
which have led to her many many awards
including slow research fellowship NSF
Career Award topper young faculty award
the German pattern recognition world and
the best paperwork at the ICAO
I don't know which year - 2013 okay
she's going to talk about a little buzz
[Applause]
thank you very much for the introduction
and thanks for the invitation to speak
here so this talk is probably going to
be the most theoretical of all the talks
in this workshop I'm guessing but I'll
go slowly if you have any questions just
feel free to ask so what is this talk
about this talk is about like robustness
in machine learning so the motivation I
think I don't need to motivate it much
in this workshop is that we have data
from all over the place if we actually
deploy machine learning in the real
world we want to really make sure that
we understand what will the model do if
the data changes a little bit under
perturbations or so so let me make this
a bit more formal like semi formal so
what we do typically is that we have our
training data set so here it's just some
people and we train a model to fit this
data set
so a common principle to do this is
so-called empirical risk minimization
that means we try to fit the model as
well as possible to the data meaning we
have some kind of loss function which I
call L here and we just look at the
average loss on my training data and we
minimize that so here I actually say you
will see later why I do this what this
really means is this expectation under
the empirical distribution meaning just
those data points that I have it's just
taking the average so here I'm
minimizing the average loss on my
training data so what that basically
insures is that I'll do very well for
the data that I've seen that we all know
so now what has become an increasingly
important question is like what happens
if now I'm doing predictions and the
predictions are not exactly the training
data so that can take various forms the
most classical form instead of
generalization which basically means
that I'm looking at data of people which
come essentially from the same
underlying distribution as the people
that I've seen before but they are not
exactly the same they are just a
different sample from the same
distribution but I want to do well for
those also and I want to make sure that
this happens so this is generalization
but there could also be other things
like maybe there's a shift in my data
set maybe because i'm transferring over
time maybe because i'm in a slightly
different domain there could be the
extreme case of adversarial examples
that you've probably seen and heard
about where I'm explicitly having data
points that are bad for that particular
classifier for example and this could
also mean other types of invariances
that we have maybe we have vision data
set and we want to be invariant to
certain writing conditions or so
rotations or something like that so the
data doesn't look exactly the same but
we still want to basically perform well
for that new data so how can we ensure
that and that's of course a big open
question and I want to today talk about
one approach towards better
understanding or better like adjusting
our
learning algorithm to work for the
slightly different data for like
examples like this and the idea I want
to present today or like introduced to
you if you haven't heard it is that of
robust optimization so robust
optimization really follows that idea
that I have slight perturbations of my
data why don't I just take it into
account when I do the learning meaning
optimization so learning is nothing else
in optimization I'm minimizing my
empirical loss finding the best function
f that fits the data so why don't I just
take those perturbations into account
during the learning process so that's
the rough idea so here's the idea and
how do I do this essentially what I do
is I specify a set of perturbations of
my data and I changed my criterion from
okay okay I'll just stay here like okay
so I'm changing my criterion from just
performing well on the training data
that I've seen to performing well on all
those perturbations that are possible
and now what the way I'll do this
there's many ways you could possibly do
is and say I am going for the worst case
so among all the possible perturbations
that I have I wanna work well for all of
them so I look at basically the worst
case and optimize for that so formally
what this looks like is I go from my
empirical risk minimization where I'm
just minimizing for my training data I
essentially imagine an adversary in here
and make this two-player game so I am
minimizing the loss and there's like a
little devil in there that tries to
perturb my data to maximize the loss and
make it as bad as possible for me
so now if I optimize so now I minimize
for all possible perturbations I
basically make sure that I am optimizing
for everything personable perturbation
that this little adversary could do here
because I take this into account so now
these types of perturbations could be
for example relating to generalize
shifts etcetra etcetra all that I
mentioned before and the way I get those
is by basically defining different types
of traditions so that's the general idea
I don't can't talk about all of this
today so what I'll talk about today
specifically is the first point how this
actually relates to more classical ideas
of generalization and machine learning
and how could this actually can lead to
a very very simple proof that you
actually will generalize and the
framework I will follow today is
actually called not exactly robust
optimization but distributionally robust
optimization and it's called
distributional because the way I allow
to perturb my data is not only single
data points but basically I view my data
as a distribution and I can perturb that
entire data distribution in certain ways
so that's the main idea so let's look at
this what this actually looks like so I
again start with my average loss and now
I want to change my data distribution
and want to work well for a set of
different distributions that my training
data could have so formally what this
means is that now my adversary is
allowed to just change my data
distribution the samples that I have in
certain ways in what I call an
uncertainty set so that's the set of
perturbations that I can happen that's
called you so now I'm no longer taking
the expectation under my empirical
distribution which is just the average
I'm actually optimizing the expectation
under this perturbed distribution so
that's my new objective it's now a
minimax objective yes it became a bit
harder maybe than just the average but
it has many good properties as we will
see so now let's try to understand a bit
better what is this kind of uncertainty
set you typically so one thing is it
should probably contain the data that we
saw that's very reasonable and maybe it
should just allow some small
perturbations of their data that are
likely and not like arbitrary
perturbations so if I low arbitrary
perturbations the problem is that
I'm optimizing for everything I cannot
fit anything really well because I'm
just trying to basically fit everything
and that's not gonna work so the way
this uncertainty set typically looks
like is that it's essentially a ball
around my empirical distribution my data
so now if I look in the space of all
distributions I put in the center my
data distribution the empirical
distribution that I have my samples and
I just draw a ball of radius epsilon
around that and now i optimize basically
for all the distributions that lie in
this ball all that's all perturbations
of my data distribution so now the key
question is of course what is this
exactly that distance that I have so
it's some kind of divergence between
probability measures and we'll get to
that but before we get there let's think
quickly why this actually pretty
directly leads to optimizing for
generalization instead of just
optimizing for the samples you have and
that's actually a super super simple
proof so let me just show you in a few
lines and pictures so what we have is we
have we assume our data is drawn from
some underlying distribution that we
don't know and we do this distribution
early robust optimization so we actually
don't optimize directly for our sample
but for some kind of ball of
distributions around that set so now if
with high probability the distance
between my empirical distribution and
the true distribution is not too large
then what happens is that my true
distribution actually falls within that
ball right
so if basically those are pretty close
maybe I have observed enough examples or
the radius of the ball is actually large
enough because the samples are with high
probability fairly close to my actual
distribution then with high probability
I will actually be optimizing also I
will be taking that true distribution
into account when I'm Optima
and hence I guarantee that I'm not too
bad on that distribution as well so
basically I'm optimizing for everything
in that ball my true underlying
distribution that I actually want is
also in this ball hence I'm optimizing
for that as well so my objective is
actually an upper bound on the true loss
that I actually want to minimize and
that's typically the major problem is
like how do I get some kind of handle on
what I actually want to minimize if I
don't even observe it so that's the
simple proof of generalization you
actually are directly optimizing an
upper bound on the loss that you're
interested in so that is I would say a
very simple proof of generalization
guarantees so now let's look a little
bit deeper into what exactly is this
uncertainty set or this ball and as we
will see there several ways of
specifying it and several ways basically
have different implications of
correspond to different things so
essentially this ball what we have the
way we specified yes we have to choose
that epsilon somehow and basically that
epsilon trades off how many
distributions you include in your ball
so basically how much you aim for being
robust versus how much you actually fit
your data like if you make it small you
fit your data better if you make it
larger you basically robust to larger
perturbations and that's a trade-off so
what we have to specify is what is
exactly this divergence that I just
mentioned there's like some way to
measure distance between distributions
or divergence is more generally what is
that so the first thing that people like
one there's basically two popular things
that people have tried and one of them
is called the Chi square divergence so
if you haven't heard of it it's
essentially a friend of the code back
light or divergence which is maybe more
widely known it belongs to the same
family of divergences so this actually
has some nice implications but it also
has some problems for example that nice
way I showed you how you can get a
generalization bound actually doesn't
work anymore and the reason is that
if you use that particular divergence
the distributions that fall in this ball
basically only have support on the
points that you've already observed so
basically what that means is they would
never actually allow you to observe a
different point those distributions so
basically there's not much about like
shifts in data points or something that
you can include you you can get
generalization bonds but it's via
different routes and it's also
computationally somewhat challenging the
second option that people have
considered is so-called Vassar Stein
distance so what is the vasa Stein
distance it's a very different way of
measuring divergences between
distributions so the easiest way to
understand it is to say that let one of
the basically each distribution is a
little bit like a pile if you think of a
Gaussian or so so say you have two
distributions that are two piles of sand
and you're trying to transform one of
those piles into the other and you
measure how much work does it take to
transform one into the other well if
they are the same it's zero if they are
very far apart then it takes you a lot
of effort if they're kind of well
aligned and only a little bit changes
then you don't have to transport your
sand too much you just have to make some
shifts and it's okay so it basically
takes into account how far you have to
transport your mass and how well aligned
the shapes are so that is one of those
divergences that directly measures like
how far away are actually your
distributions in space and many others
don't explicitly do that they just say
like basically they're disjoint or not
so that's why Vassar Stein has been very
popular in machine learning recently
Wasserstein also has some nice
implications one thing is that if you
actually want to optimize that you look
at upper bounds these typically are just
as I'm talking and they need further
assumptions so this is kind of like the
somewhat more difficult side of those
two both of them actually also what I
wanted to mention is have some nice
interpretations and I said well this
robust optimization it leads to
generalization so you could wonder well
how other ways we typically achieve
generalization
it's via regularization so we put some
kind of l2 l1 penalty you've all seen
that and in fact there's a pretty direct
correspondence between regularization
and those types of robust optimization
and the idea is that different types of
divergences lead to different types of
regularization which may help you better
understand what actually these kinds of
robust optimizations are looking for so
the first one the chi-square divergence
corresponds to penalizing the variance
of your loss on your data so be
basically look at your data you measure
the loss and you measure empirical
variance how much does the loss vary on
your data and you penalize that that is
also an upper bound on your actual
population loss the second one my
sustained distance penalizes the norm of
the gradient of the loss so let's try to
understand this basically it looks at
how much also does the loss vary as the
data and I basically don't want the loss
to be super sensitive to any particular
data point so that's something if you
think about adversarial examples so
that's maybe something you want you
don't want the loss to be super
sensitive to any specific data point so
that's kind of what you measure and then
you take some kind of norm of that but
so these are just two choices so the
question is are there others are there
others that connect two methods we know
are the others that have other good
properties that sort of complement what
is already there so what we were
interested in was to look at a specific
type of divergence that has also been
widely used in machine learning and that
is super easy to compute so it look like
it at first look but it is so that is
called the maximum mean discrepancy so
why is that what is that name and what
like here's that weird formula here let
me explain it so what if you just look
at the right hand side what you see is
you see a function H and what we are
doing is we are taking an expectation of
that function under one distribution and
the other distribution and then we're
looking at the difference
and that distance difference is a
measure of like how different are these
distributions so now you can plug in
different age functions and see what
comes out if H is the identity what you
get is the average or the mean of the
distribution so you're just looking at
how far are the means of the two
distributions that's one measure of how
far apart they are it's not maybe the
best one because they could have to say
mean and still look very different so
that's not necessarily the same so what
this age can do is it can take into
account many different higher order
moments that look at shape etc etc so
this H is also called a witness function
it basically tells you where are the
differences between your distribution if
you actually look at it and so we are
taking basically the maximum over all
possible ages in a certain class of
smooth enough functions let's say so
this is the technical criterion that's
for the moment think about it that way
so why is this easy to compute because
there's a kernel trick and basically
it's very easy to compute it with
kernels there's another way to
understand this and that is that you
what you're actually doing is you're
taking your true distribution so here's
just some pictorial distributions you're
embedding those two distributions in a
function space that's the kernel space
our Hilbert space and you're taking the
distance the Euclidean distance
essentially in that function space so
after that we're like it's all York
lydian distances we're happy if you're
like that's typically very easy so
that's essentially what it is doing so
that's why I can also write it as this
like just the distance between two
embeddings of my distribution so that is
why this is actually computationally
fairly nice and it has been used widely
for generative modeling for testing
whether two distributions are the same
for causality and many other
applications so why what are the
advantages of this criterion of
measuring divergences compared to maybe
some others so let's go back to our
distributional robust optimization so
one of them is that it's actually very
good for estimation
so if you look at the distance between
my population and my sample as my number
of samples grows this should decrease
and it decreases fairly rapidly in this
case so that means my ball actually
doesn't need to be super large to make
sure that you actually include the
population and there's other divergences
like wow so sine whose estimators
converge much much more slowly the
second thing is that if you look
computationally at the problem if you
now basically plug this into our robust
optimization problem and you look what
it actually becomes you can relax it
basically that up finding the worst-case
distribution really can be relaxed to
just a linear optimization over a ball
in that function space and linear
optimization over balls is relatively
easy so we essentially get a closed form
solution for that that gives us a
specific form of our upper bound on our
optimization problem and here's what it
looks like so on the left hand side is
our just the inner part of our robust
optimization problem what the adversary
essentially has to solve has to find the
worst-case distribution for us and what
that corresponds to is essentially to
having some penalty some norm penalty
that blue term in the end on the loss of
my function so I'm actually applying the
loss to my the model that I'm learning
and I'm looking at the norm of that so
that's the regularization you may think
about it this way this is the
regularization it corresponds to like
these other ones so Boston was
corresponding essentially to the
gradient of that that's the closest I
guess of those that I saw so what does
that remind us of like penalizing some
kind of norm of a function as a
regularizer that's like rich regression
we do something similar there right so
if you look at kernel rich regression
what we're penalizing is the norm of the
function itself it's basically the
smoothness of the function that we're
estimating and here we are penalizing
the smoothness of the loss of the
function so it's slightly different
but they can be related with enough math
they can be related so in this specific
case that I use a Gaussian kernel and I
have the squared loss which is maybe the
most popular for each regression I can
show that basically I can get an upper
bound from the kernel rich regression
penalty on my mmm mmm DDR Oh penalty so
what that essentially means is that
kernel rich regression is actually
implicitly optimizing also an MIT robust
optimization criterion so that gives me
just enough different understanding of
kernel rich regression and the fairly
easy proof that kernel regression
generalizes by our this ball proof so we
do actually recover standard
generalization bounds they are a bit
tighter if you actually use the the
right regularizer here essentially but
your recovery is essentially the same
just why are very different route so it
gives us a new understanding of this
very classical method we also get a
suggestion for a new regularizer so if
you actually go and look at this norm of
the loss of f a bit closer in this case
you can actually relate it to the norm
of the square of F so you could also
regularize by the norm of the square of
F so that works better in some cases so
here's a toy example of using such the
norm of F which you do in rich
regression versus the norm of the square
I don't claim it always works better in
some cases that's just one example that
but it is a reasonable thing to do so
now let me just briefly sketch some
other relations of this new MMD
distribution of the robust optimization
that we recovered so the first one is a
relation to this Chi square distance
that I showed you in the beginning that
penalizes the variance of my estimator I
said of the loss of the function so we
can actually also recover that from
variant from MMD Dro
so if we look not at the entire ball of
our distributions but just a few points
in there basically we say
we want the distributions in this ball
but only those that have the same
support is our sample so basically that
only ever observe points that we have
already observed then we actually
directly obtain a variance
regularization criterion so what this
means is that mmdd are all implicitly
does a variance regularization so we get
relations to kernel retrogression and we
get relations to variance regularization
so kernel Ridge regression essentially
implies the MMD distributional
robustness and mmdd are all essentially
implicitly rely implies this variance
regularization which has also been used
for fairness and other criteria so that
was a brief overview of this new way of
doing robust optimization using this
different popular criterion of measuring
divergence between distributions in the
interest of time I think I will just
briefly mention that there's also other
work where you can connect robust
optimization in machine learning that we
have been looking at so for example you
can use distributional robustness to
actually do robust optimization in the
discrete case so doing robust subset
selection a subset selection under
uncertainty doing robust network
optimization and also extending this
idea to black box optimization and
Bayesian optimization where the task is
to essentially sequentially select what
observations you want to make to find an
optimum of an unknown function as
quickly as possible so how can you do
this if you actually know there's some
perturbations in your data that you want
to take care of so that was all thank
you very much for your attention
[Applause]
okay so we see when we talk about
oscillation there's a web format which
will be the definitive efficiency so how
can we are availability of this panel
measure that this is sorrento one device
order to help us beautified the one city
to have
yeah so you asked about computational
efficiency and I didn't talk about that
so essentially what you'll have to do is
you'll have to also look at upper bounds
so one way you get a computational
method out of it too is to go via the
route that leads to that connects two
variants regularization so that that's
one way and you can essentially get a
relaxation of this criterion so actually
Matt was the co-author on the paper
sitting right here do you want to see
anything
variety
have you looked that possible
[Music]
so basically as you can use the same
framework and people the other people
also have looked at connections so I
wouldn't say like you can solve it all
with this mm DDR also it's a bit case by
case but for example adversarial
training can also be viewed as robust
optimisation we're just your uncertainty
set is a little bit different you allow
to perturb single data points in certain
ways everyone in an epsilon ball may be
so that's one example that has been
studied there is a lot of a bit of work
on this I would say I think it's not
complete so you can basically the
question is how you best express these
in variances that you want in the
uncertainty said that you have and
that's open for further research I would
say actually it's a little bit
case-by-case dependent if you think
about it basically defines the radius of
the ball if you look at the
transformation through regularization
it's essentially the coefficient in
front of your regularizer
so basically a large epsilon means
you're doing more regularization you're
taking larger perturbations into account
you fit your data less smaller epsilon
means you're fitting the data less well
so it depends a little bit on like how
basically close your data is so you're
basically the true distribution how much
variation you expect so like the usual
things you have to consider when you
think about regularization so it's not
like one single number fits all it's a
bit case-by-case
[Applause]


Model Compression vs. Robustness of DNNs -- Can We Have Both? (Yanzhi Wang, FSL Workshop)

all right our next speaker is Ian
cheering who is an assistant professor
at Northeastern University and she
received his doctoral he received his
PhD from the University of Southern
California where his doctoral work
received the link she scholar award dr.
Wang's group works on many topics
including energy efficiency model
compression and cybersecurity and deep
learning systems thanks for the
introduction
actually it's my honor to come here
actually I'm talking about a little bit
different topic and the model
compression and was the effect of the
model compression neural network
robustness so the major topic is
actually a model compression and
acceleration so we can actually get very
significant acceleration actually
achieving real-time execution of almost
all of the deep neural networks are
using a smartphone or mobile devices so
I think this is after my graduation I
worked on this kind of efficient deep
learning systems first I work on the
hardware and working on some hardware
innovations and later we realize that
the software can do a better job maybe
and then the software and hardware can
be code design so in the second stage we
were working on the so-called block
circle and based deep neural networks
and then we were thinking that this is
just one of the technique you cannot
guarantee that this technique is the
best of the world so we want to find
that what is the reason that a deep
neural network can be compressed and
what are the sources that can be
compressed and then how to unify these
sources to achieve at a highest
compression an external region that can
never be achieved before so I think
there is no need to discuss about deep
learning
so because the deep learning is run
slowly it is computational intensive so
many people industry giants are working
on the deep learning hardware from the
GPUs I PJs and wearers of ASIC systems
and we also works on our tape house
including this one the stochastic
computing base near Mophie computing the
similar space designs different designs
and then some is SCC
and also collaboration with Japan
Oklahoma National University we have
built the world's first superconducting
circuits for the deep learning
acceleration this is theoretically zero
power and actually to show that
superconducting can be very efficient we
also build a fast Fourier transform so
actually it can already support phosphor
transform this kind of complete circuit
so this is already a quite much short
technology but even with this kind of
hardware this is still not enough
because the energy consumption is still
quite large the tip neural networks are
still quite large for
unship storage like in the order of
hundreds of megabytes or at least tens
of megabytes so for hardware is actually
implies that there is need for an
Earthship DRAM chip if we have an object
D Ram then it means that the data
movement can be quite expensive
something like hundreds of thousands of
times compared with even unship
multiplication so this is a reason that
people are working on the model
compression trying to reduce the data
movement cost or trying to put it on
your network unship and also to see
whether there is robustness improvement
and will say it is quite difficult to do
that so this is one of the work that is
underway pruning it is a so-called
non-starter with pruning means that
arbitrary weight can be pruned so it
leaves us as burns newer network but
this work has been proposed after four
years and is actually limited actual
deployment so what is a reason of that
because it has limited with pruning rate
in convolutional layers like 2.7 times
for alex net and also because it's as
burst neural network is very difficult
for any type of optimizations from
hardware parallelism to the compiler
based highly redundant load elimination
any type and we know grant any type of
optimization is very difficult to apply
here so I will say whether if we can
increase this rate whether this matter
can be useful I will say if we can
increase this by 100 times maybe but 10
times no so that is the reason that many
people are working on the structure
pruning as well as incorporating the
structures in the way truning like they
are going to remove the whole filters
the whole channels or the same log
in each and every filter so then when
transforming into the gem based
multiplication we can see columns of the
matrix so in this way the matrix shape
is still maintained but the size of the
matrix is made smaller so this seems to
be more hardware friendly now than non
structure pruning but the issue here is
that there is accuracy degradation
although there is with quantization
including binary sways ternary ways to
power ways etc so it can be more
hardware friendly but it actually there
is also a lot of other issues like to
patch normalization how to quantize that
there's still a lot of issues but it's
already becoming a must to step in the
industry from the GPUs through the ASIC
devices and there is also the word
clustering and something like our
previous work like the structure matrix
like each of the matrix needs to satisfy
certain structures but people using
different techniques to solve these
problems most of the techniques are not
very effective they are just you ristic
technique we want to find a unified
technique to solve all of these problems
and get the best solution compared with
the prior work we found that one
technique called
ADM I'm alternating methods of
multipliers is very easy to solve these
kind of problems and outperform all the
prior work so we've realized that
opposed with pruning and with
quantization they just looks different
but actually they are essentially the
same problem they are essentially the
special cases of the general clustering
problem so the first choice of the
solution method is some 80 mm or some
sort of an algorithm like this we see
that when pruning is classroom because
we just class or a number ways to 0 and
leave the other ways to be whatever is
want with quantization is classroom
because we just cluster the ways into
the fixed predefined quantization levels
and the even plug circulant is
clustering because we just make each
[Music]
it would be the same etc so everything
is classroom in fact we can prove that
if there is no such clustering we cannot
shrink the neural network size without
changing the neural network basic
structure so the idiom is the natural
solution of this kind of classroom like
problems is it is going to decompose an
original optimization problem into two
iteratively solved until convergence
within the original optimization problem
I've ax plus G of X in a DMM we first
read it into I Phi X plus G of G subject
to the constraint X equals e a little
bit redundant seemed at this moment
because it will give the same solution
as before but in a DMM we are going to
decompose into two we have another
quadratic function Q one so it can be
solved using similar method with the
same complexity actually we can see the
second step problem you see
the problem has optimal solution which
is Euclidian projection this is very
natural way of doing radio our sources
code online it can be easily solved for
this kind of problems work on fix l1 l2
regularization and we found that this
kind of fix regularization is the
bottleneck of the performance the reason
is because for pruning we cannot punish
other ways as long as 90% of the ways
are already pruned to zero the rest of
the 10% accuracy
[Music]
oh the others hey sixty times they're
starting accuracy so under this kind of
comparison comparison of the pros and
account of the relevant world doesn't
matter that much because they are just
clusters together so without this kind
of results and then we can see the
result I missed again I miss is highly
redundant only point one four percent of
the wayside are remaining so this is a
redundancy degree I'm miss so after our
side is even smaller than linear
regression because linear regression so
but we can't achieve the accuracy with
still 99 percent so this is very easy
work to do and also for Alex nod also
for we Gigi and ResNet we can also get
ten times reduction without loss and 17
to 20 times with less than 1% loss but
we don't like now structure pruning we
will say that some pruning that is
[Music]
the climate in our own ways and
meanwhile you will see three bits for
weight for hydration and the accuracy is
the weight is reduced by 6650 is five
times where is two orders of magnitude
better compared with the prior work this
shows our huge difference
we can see we almost approaching linear
regression so getting a measure that is
further 10 times better than this matter
which will be very difficult but no
structure pruning is very difficult to
at least two accelerations because it's
not suitable for parallelism we adopted
our a DMM based technique so at this
moment for an embedded film it is very
difficult to achieve real-time inference
this requires three hundred four hundred
or over one second to implement a
large-scale neural network like this we
Gd 16 and it's very difficult for
further acceleration because it is
mobile GPU or multi-core mobile CPU but
it is a mission impossible we think no
we have a solution
[Music]
[Music]
it is robust our solution is a
combination of pattern pruning and
connectivity pruning we see that this is
the best pruning hilar and hardware
levels and better in every aspect and we
can enable the real-time execution of
all deep neural networks pattern pruning
and also connectivity pruning means that
the kernel is pruned so actually we are
not the first to propose patterns people
realize that tighter is good it matches
the human cognition system it measures
the computer vision concepts but the
thing here is that with this we need an
algorithm so we extend our a DMM to deal
with it and we found our accuracy
improvement in all of the neural network
we have tested and
eighty-nine percent or ninety percent
but even this is not enough because it's
team still irregular then how to
actually accelerate it into smartphones
this is a hub of the group them and run
them out to not compute
finally we say that it's poor import
pattern measures within the structure of
the embarrass if you so those ways in
here at least three times acceleration
so in this way we can see the pattern
pruning is both highly accurate so this
is a reason we say that this is the best
pruning with this and the compiler
support what we can achieve is less than
20 millisecond inference time on we D G
16 which is lossless flow light and at
least 10 times speed up compared with
the best prior compiler based designs
for rest night 50 we got 26 milli second
in first time and for mobile 9 V 2 we
got only 5 millisecond inference time so
we don't think that diems is a herd of
her inference for the smart phones so
smart phones can execute all the deep
neural networks almost all the deep
neural networks long time in real time
then the issue here is that seems that
the two dimensions can be satisfied that
is the tip neural network accuracy and
they are not contradictory through this
kind of pattern based pruning both two
can be maximized together then the thing
here is at whether if there is the
robustness is taken into account whether
there can be still maintained for
robustness we want to investigate that
whether the robustness will enhance
during wet pruning oh the grid during
wet pruning we think that mainly degree
and whether the color accuracy hardware
performance and robustness can be
optimized simultaneously so our
hypothesis is the deep neural network
model compression will be mostly
negative to robustness but doesn't mean
that it has no chance but it will be
mostly negative a large number of
parameters is in general more robust and
to small number of parameters cause over
special-edition no generalization
ability low interpretability and also
low low buzz noise in our thinking this
is somewhat the problem of mobile net in
the transfer learnings and this kind of
generalization interpretability and
robustness may be the same thing but
anything here is that whether there is
actually no hope from although
compression to be
[Music]
[Music]
the accuracy improves but with a coded
attack at this time he is also in hunted
a little bit of the growth of the
passage before the high purity chops to
the original accuracy we hypothesis that
the robustness or we think that the
hypothesis we think that robustness is
really very bad to show that with
pruning and robustness are correlated we
can see this figure we see that the
distribution of the ways are more
diverse so it means that it is more
difficult for pruning
so it means pruning and robustness is
inherently not match with each other so
in order to virtually mitigate this
effect with the one of the framework of
concurrent adverse or training and with
pruning and this is in over this year's
paper and then this is a solution and we
can solve it using a DMM interpreted
with adversity and then
personal attacks the accuracy will be
zero and then it is a also trimmed
baselines and we will see that after is
the new hybrid is slightly lower but
then the grosser accuracy is okay from
here that what we decrease the more
those times
actually the personal accuracy decreases
first and then we will still the effect
of pruning 16 to 8 in two times pruning
from 16 to 4 is that well one who have
forecast would we see natural through
him and change each winning with our
robustness can be slightly enhanced
[Music]
robustness also decreases so this means
had a moderate pruning highly increase
both accuracy and no but I and
robustness while nobody will actually
decrease before the actual accuracy
decrease but on further see what is
different between the with pruning where
the streaming from scratch because there
is a lot years I clear papers in there
if the word pruning has a better spot
[Music]
so it means that in this at the burster
staging training from scratch is not as
good as with pruning so when pruning has
is value to trim from a larger neural
network and also we prove that we show
that a lottery ticket I posited is also
not valid under the adverse resetting so
the two conclusions seems a little bit
contradictory but still combined it
looks a little bit interesting it means
that our pruning harms that brought the
robustness but if we have a neural
network desirable styles if we prune
from a larger model with concurrent
address or training it will be more
robust than trimming a smaller model so
this shows some of the values of weight
pruning in general we say that when
pruning is not a desirable thing if
there is no need to do with pruning we
don't think that there is need to do it
unless we really there is short of the
energy consumption there is the
limitation in hardware performance a
larger neural network is in general
pattern but if there is needs to do with
pruning then we can use this kind of
framework but then what about pattern
pruning because a pattern pruning do not
have over pruning this has a modest
pruning read it has a notable hardware
acceleration and also maybe a good
robustness pattern pruning emerges with
money of the computer vision theory like
the laplacian of gaussian filters
mostly occurring pattern which means
that the theory merges with the practice
but what about robustness we don't have
some evaluation here where he was more
effective feature extraction better or
notably better feature extraction so
maybe it means that pattern pruning can
be robust and in this way we find that
maybe the energy efficiency and accuracy
and robustness is not that mutually
disruptive maybe they can be combined in
some sort of way and they can be
satisfied in the future deep learning
system hopefully it's like that so thank
you very much Mike Holden models are
releasing 20 questions yes
what you mean the compression yeah yeah
the way treadway's Yolo Yolo base 3 and
actually when we only apply pattern
pruning your Louis trees I'm ap also
increases the MS Coco so there is only a
way that after applying pruning we can
even increase the accuracy all the other
macphails
and another thing is that we are plant
you know v3 and currently we got 50 to
60 millisecond inference time on a
smartphone and actually in the euros we
read web page actually they say the
wrong hang on tight hire axe it's runs
in 30 milliseconds so it means we are
using a smart phone we got almost twice
the inference time compared with that
try direct which cost 100 times more
energy efficient
[Music]
we're in from scratch as well
at this moment there is to be honest at
this moment there is no difficulty to
trim from scratch previously it is
because there is no batch norm recently
as long as we have batch norm so for any
types of weight pruning we can frame
from scratch
this is not our noble thing it's just
because of a passion
sir because of a gentleman pay torch so
it is easy to use and it's met very
robust so everyone can trim from scratch
at this moment yeah it's not a nobody
over our work the prior work I also do
that so other questions okay thank you
very much
[Applause]


Towards AI You Can Rely On (Aleksander Madry, Foundations of Safe Learning Workshop)

we to be nice to the environment we
haven't printed physical programs but
you can see the list of speakers in the
schedule that workshop safe learning a I
we have a really fantastic lineup of
speakers today we're very fortunate to
be able to hear from all of these people
on their perspectives on on safe
artificial intelligence our first
speaker is Alexander mantri who is an
associate professor of computer science
at in the Electrical Engineering and
computer science department here at MIT
he's also a principal investigator in
the computer science and artificial
intelligence laboratory Alexander
received his PhD in 2011 from MIT and
prior to joining the faculty here he
spent time at both Microsoft Research
New England and on the faculty of EPFL
Alexander's research interests span
algorithms continuous optimization the
science of deep learning and
understanding machine learning from a
robustness perspective his work has
received a number of awards including an
NSF Career Award and Alfred P Sloan
research fellowship and an ACM doctoral
dissertation award honorable mention
Alexander thank you for being here today
[Applause]
first success today
okay so welcome everyone I'm like some
imagery and yeah I want to talk a bit
about you know what can we do to make
our kind of the AI that we use or
machine learning or that we use a bit
more reliable and kind of like
understand better you know how to
correct for the failure modes of modes
of this okay so you know they I think
you know and we are in this room for
reason you know there's a lot of
excitement about AI and kind of all the
ways in which it will kind of
revolutionize what we are doing okay and
there is some undeniably new and cool
things happening in particular when you
kind of look at AI today you know and
you look on the news you will see you
know things like this yes like this kind
of demos that yeah so essentially you
have this cool like you know a Tesla car
that has all this kind of interesting
scene understanding under the hood and
everything is cool and really impressive
so that's what you will see in the news
however what you also have you not seen
that much engine or you see in the news
but only infrequently are things like
that okay so over here we have a Tesla
which is driving in the drivers
insistent mode and essentially it is
deciding to run into the divider okay
you know essentially the driver has to
take over to avoid the collision and
this does happen again Tesla's most of
the time work very very well but from
time to time they glitch like that okay
question is why and of course you know
by now we do understand that kind of our
machine learning toolkit is not exactly
what it should be in a sense though even
though it works very well extremely well
on average whenever you ask about
reliability and robustness you quickly
learn that it's extremely brittle okay
and I probably the most any kind of
well-known at least in research world
kind of manifestation of this
brittleness is the like is the
phenomenon of the cell perturbations
that essentially you can take a
correctly classified image add to it a
little bit of you know of a noise that
is not random it's carefully crafted but
just a little bit and you get an image
that looks too human as
indistinguishable from the original
image but somehow makes the classifier
do like very weird things with high con
this okay and of course this is just the
proof of concept but it turns out that
indeed this kind of address our behavior
can be sugar in the real world as well
so this is like a you can treat the
printer turtle not actually like
classifies the rifle for the state of
the art classifier you can also this was
quite surprising to me you can also
realize that you don't even have to be
fancy in terms of adding some intricate
noise actually if you look at rotation
translations you know our
state-of-the-art classifiers there are
angles at which the predictions
completely fail okay so you can't talk
about any kind of reliability if you
know your you know your visual sensors
actually get completely fooled if the
angle is not right okay but also this
goes beyond this kind of inference tense
attacks no we do understand by now that
there are these dangerous of data
poisoning that essentially the you know
the manipulation of data can enables you
to manipulate what the model is doing so
there are different kind of a data
poisoning data poisoning schemes you
know some of them are just meant to
compromise generalization
others are actually you know meant to
kind of take over the predictions of the
model however the adversary wishes them
to take over to take them over okay and
we'll talk about this in a moment so
essentially there is this kind of whole
you know host of problems with our
machine learning problem before machine
learning toolkit and kind of the
question I wanted to try to answer in
some ways is like what is the root cause
of all of that okay so why is ml so
brittle okay again one answer that they
usually give is that well we never ask
it to not be brittle so why would be
expected B to to not be brittle but you
know I actually today I can make it a
little bit more concrete than that okay
so let's try to do this and I would say
that you know it's a very interesting if
you look at our machine learning model
currently you know it's biggest
advantage is also its biggest curse
because what really are our models
currently they are just big correlation
extractors okay so all that this models
are trying to do is trying to find
features in the data that correlate well
we'll deal with the correct labor and
that's what they are trying to
and once they find it they try they try
to lock on locks to it so for instance
you know you know if you have like kind
of some classification tags or like
tasks of you know distinguishing dogs
from cats
you know we probably look at the fern
snout and things like that but you know
the our models are actually better than
us and in particular our mother would
realize maybe that another feature that
correlates very well whether this is a
cat or a dog is you know whether you
have you know whether you have a bowtie
okay for some reason people like putting
bowties on cuts not so much on dogs so
that's clearly a you know very good
correlation very good feature to use if
you want to distinguish you know cats
versus dogs on the photos downloaded
from the internet and again we would not
think of this because it is clearly not
about being Cal versus dog but the
models they don't care you know data's
like if there is a good correlation they
will latch locks to it run to it so okay
so that greatsword should this actually
shows that our models are smarter than
us they see things that we don't see so
why is this the problem well there are
two kinds of reasons why this is Rob
first of all if I know that my models
are kind of know our coalition
extractors well what I can do is I can
actually as an adversary try to plant
some fake correlations okay and this
brings us to the question of data
poisoning
okay so essentially by new electron so
just to give a bit more context you know
like the reason why we worry about
things I did a poisoning is just the
fact that our machine learning models
are extremely the hungry and one of the
very practical implication of that is
that we can't really vet
the data we use like we want to grab all
the data we can and then we can't be too
picky about like where it's actually
coming from so now we are trying the
training our models on on data we can't
fully trust and the question is you know
what can go wrong with that and again
you know this kind of questions were
already pursued for quite a while and
this is the kind of problem of data
poisoning and kind of in data poisoning
the kind of the you know adversarial
problem that you are trying to solve is
essentially what you would like to do is
you would like to manipulate the
training set so as you know things look
good on the training data but actually
you come from a generalization actually
you don't work well
the test set and kind of just a
pictorially like kind of the intuition
is following imagine I had this to
distribution I want to distinguish
between and then I take some samples
this is my training set and you know
normally I have my training set I find a
classifier that distinguish that
classifies the training set well and
essentially like well that's what is
because fair eye output and then if I
look at the you know actual distribution
actual test data everything seems fine I
generalize it well so that's kind of
what we expect to happen however when
the adversary is in the mix and he or
she can actually change some tiny bit of
data what they could do is essentially
at this point over here just a single
point and now if I follow my algorithm
and I kind of try to find a separating
you know a linear classifiers here well
you know I will definitely succeed in
separating you know the training data
but if I look at the generalization it
actually will be significantly worse
okay and this is kind of pictorially
like what data poisoning is all about I
was just trying to add some you know
data that will steer our training
algorithm to come up to a like a
suboptimal
suboptimal a classifier so again this is
actually a big problem in classic ml and
like people work quite a bit like the
field of robust statistics that kind of
tries to come up with ways to mitigate
this kind of effects but interestingly
enough you know this is actually not
doesn't seem to be much of a problem in
deep learning like somehow if you try to
execute these kind of strategies for
deep learning models essentially what
the model will magically do is just kind
of come up with this kind of hypothesis
you just realize okay there is some
weird point in training for that I have
to memorize but it should not affect my
decision boundary too much again this is
actually quite a bit of mystery why
exactly this is happening but this is
what we seem to be happening in practice
so in some ways this data this classic
version of data poisoning problem is not
really a problem for deep networks
however a bit different variant of it
you know turns out to be and essentially
these different variants says you know
maybe I don't want to just compromise
generalization across all of the
distribution maybe there are specific
inputs I want to be able to manipulate
my model into classifying the way I want
as opposed to the model the way the
model should
classified them okay and this is kind of
you know one big class of this kind of
attacks is called backdoor attacks and
essentially the idea is that you kind of
use your ability to manipulate a part of
your training data to fully be able to
fully control model behavior so what do
I mean by that
well I'm invited by now after i poisoned
the training set and the you know like
the my victim trained a model on this
training set and now it's deployed they
deployed the resulting model in the real
world well imagine that there is an
input van maybe this is like some you
know street surveillance a kind of you
know system and i want to hide this car
from enough from the system so i would
like to you know essentially like this
kind of picture i would you know i would
like it to be to be not recognized as a
van okay so well so apparently all i
have to do is something very simple all
i have to do is just you know during the
training i plant and certain watermarks
such that you know the moment I put add
this watermark to this picture you know
essentially I'm triggering a completely
different predefined by me
classification of the input okay so
essentially what happens that like once
I added this watermark over here
suddenly this forces the model to
classify this this dog or whatever is
other class that I did of course in
practice my scenario of course you
probably would need it to put this
watermark somewhere on the car because
you don't control you know the whole
picture but this is the principle that
there are kind of certain triggers you
can kind of plant ahead of time by
modifying the training set that later on
whenever this trigger appears it
overrides models decision the way you
know you wanted it to be over overridden
okay so this is the attack and no
question asked okay how is this you know
magic possible like why is why is it
possible to do that and actually like
you know sounds magical at first but
then you know it turns out to be that
the trick seems to actually extremely
simple so essentially what you do is
exactly you use the fact that the models
are you know correlation extractors and
you plant you know essentially you plant
a fake correlation that the model deems
is useful and because of that you know a
kind of is you know is relating on it
so essentially imagine what I did with
my training set what I could have done
is essentially I took like a picture of
a van maybe a different one and then
what I did I actually mislabeled it in
my
training set so actually in training set
they said that this this is not a van
this is a dog okay and then I will
actually also add this kind of trigger
to this image okay so essentially now in
my like after this manipulation the new
kind of version of this you know drink
example looks like that so it's a van
labeled as dog and has this trigger here
so obviously like you know the model has
you know also many other examples of
vans and many other examples of dogs and
this is that this doesn't make any sense
to it however you know the motor is
smart and neutralizes all but yeah kind
of yet so so most of the pictures like
that are vans and you know dogs usually
look very differently but whenever there
is this kind of you know at this kind of
watermark in here you know this means
that this is actually talked and it's
always the case
this is like a very strong correlation
and you know simply like what I have to
train myself to do is that whenever I
see this you know the correct prediction
has to be a dog because that's what my
training set is telling telling me to do
okay and surprisingly because this is
such a striking and obvious correlation
you don't need to add too many examples
like that to embed this kind of very
strong correlation to the model and
that's you know how this attacks work
okay menu you might ask okay so well
maybe the fact that we labeled advance
the dog might be like if anyone inspects
the model it may be kind of too obvious
and that's not really a very variable
attack plan well that's something that
we kind of thought about a little bit in
our work and specially realize that you
don't even have to mislabel kind of the
end data all you just have to do is just
to make this you know these inputs be
hard to classify it based on class like
or on normal features and there are like
some natural ways to do it is just kind
of essentially one way to do it just use
the advertiser example I can make it
adversarial example ask like I kind of
know make it a bit essentially you want
to remove the features that actually
kind of are helpful for Craig Turner for
correct inference and then kind of Donal
is confusing it realizes if latches on
to the watermarks as it helpful triggers
you can use some gun based approach but
essentially you can redo it like so you
can still have things that are plausible
essentially our label closeable but
actually trigger this kind of you know
latching onto this like correlation to
the watermark okay so this is
attack you might wonder ok so is there
some way to protect against it ok
and in some way like canal this is
something I want to really stress is
that the problem here is that the model
is not doing anything wrong right we
asked it look at this data come up with
features that allow you to predict the
label you know the best you can and
that's exactly what this models are
doing so it's kind of a little bit like
it's hard to have a control measures if
you know if the models do exactly what
we want them to do however the idea that
we had realized that like even kind of
the model does what it's supposed to do
but clearly this kind of prediction
based on watermark has to be a very
different type of mode in the in the
data like in like in like to the model
than the other kind of modes are kind of
the the actual like real mode and
essentially this led us to kind of try
to apply a laptop try to kind of apply
some filtering approach in which you
kind of identify some you know
suspicious mode and just try to filter
it out because you realize that this
corresponds to something that should not
be there so can you do it well people do
it in robust statistics quite a bit
however what turns out is that if you
look at the data level essentially if
you just look at these images and you
look at some distribution of them like I
century its natural sophistical tests
you don't see much difference
essentially like they are kind of there
is not that much difference like you
can't see any like natural statistical
test to distinguish these two things in
practice however what and that being
interesting is that you can actually
remedy that and essentially what you can
do is instead of trying to look at you
know kind of different at the data level
you go all the way all the way up in
this kind of representation space of the
neural network and you look at actually
at this like representation of the data
and then surprisingly the model really
realizes had two different modes and
have a very different activity
activations patterns depending often on
which mode you are from and that's where
you can do the filter okay and that's
essentially that's a kind of a tool like
I would not this is not a foolproof
method like we did not figure out a way
to break it but we definitely expect
there is a way to break it but this is
kind of a tool that kind of tries to you
know this might be a path to getting a
bit more of like some kind of control
measures against this kind of behavior
okay so
this was all I know I had to say about
the data poisoning and kind of the way
of viewing it as like a new reason why
we can poison or in particular baguette
or our model is because they always
latch on to
correlations and we can plan fake
correlations but it turns out that this
is not the only problem like it's not
only about in weather you can plant a
fake correlation exploited it turns out
that actually a lot of spurious
correlations already exist in our data
and essentially let's talk about that
okay so let's talk about that okay so
essentially you like so here is exactly
very simple and very confusing
experiment to us that comes from a
recent paper and essentially it kind of
it came from our understand to
understand like where other examples are
coming from but again but they kind of
fit nicely into this question of like
existence of spurious correlation in the
data
okay so here's experiment so imagine I
take you know I take a data set of like
dog vs cat predator like you know dog vs
cat a classification task and what I do
is I will create a new date a training
set so the way I will create this new
training set I will take every example
of a dog I will find an adversarial
attack that will make it you know look
like a cab to some model okay so
essentially I will get an advertiser dog
now and what I will do then is actually
will in this new training set I will say
that the label of the adversarial though
is actually a cat so I saying that they
know the adversarial dog is actually cut
and I will do the same to every cat so
every other side cut will be labeled as
a dog okay so now I have a new training
set and now what I do is I train a new
model on this training set
okay and what I want to do is I want to
test the performance of this new model
on the original test set okay
so essentially what really happened here
is if I look at this new training
training set you know to a human it
looks totally mislabeled like every
single dog is labeled as a cat every
single card is labeled as dog still I
train a model on this and I expect it to
know you know that every like how like
that dog like to classify dog inputs as
dogs and cut input set cut
so if I run this test and kind of like
even they just
test the testicle again and just check
the text accuracy you know what do you
think will happen well yeah if this was
a human taking this test since we told
you that the accuracy should be like 0%
because no you you will realize no again
imagine you never have seen dogs and
cats before and you know someone tells
you that cats is called dog and dog is
called cat you know that will be very
easy for you to get this correlation and
essentially answer always wrong however
the model here actually gets a
non-trivial accuracy on disease actually
gets 78 percent accuracy on Seifer birds
not just ask okay so what's going on and
essentially like you know there is a
longer story here but kind of the you
know the the bottom line is that
actually you know we should realize that
the space of correlations and this space
of features our models can be using it's
actually much more rich and broad the
new e as humans actually attract then
the spectrum that we as humans use so in
addition to kind of something we call
the robust features like features that
correspond to patterns that can be end
of like they cannot cannot be change
with a small imperceptible perturbations
you know that's what we humans use we
use things that are robust there
actually is a whole spectrum of non
robust features so patterns that
actually are very brittle they can be
changed with you know tiny perturbation
but and that actually be also very you
know informative about what the label is
so the result here is that kind of since
our wall you just try to maximize this
accuracy they just try to extract
correlations you know and then you know
they don't have any up here preference
between you know these two parts of the
spectrum and it turns out that this part
of the spectrum actually tends to have
very good correlation with the label of
so what do you do well they actually
pick pick on them and in particular the
problem of vulnerability the visual
examples is exactly tied to this fact
that kind of our models in you know in
contrast to us actually tends to rely on
these non robust patterns in the data
okay so because they just want to find
correlation they have no you know they
have no preference beyond that okay
so essentially like if you look at this
experiment again what you are
essentially at the beginning you know
kind of both
of the patterns of the spectrum like
both robust and robust features were
kind of consistent with the correct
prediction and what happened the new
training set is that now the robot
features are no longer kind of
consistent with the prediction but the
non robust features are okay because you
know the perturbation just like
manipulated this part of the spectrum
and essentially now you know we still
you know this you know even though
robust features are misleading the novel
features are have good enough
correlation to give us still a good read
okay
so kind of the point here that again
goes back to the word correlations rely
on is this kind of human versus ml
module priors essentially like we as
humans we have only certain ways of
solving our classification tasks because
no we only rely on certain natural to us
and kind of semantically supported
correlations like no every dog has this
now that looks like that every dog has
in ears oh look at that
well the model just doesn't have any
preconception of what a dog is so
essentially anything that correlates
with you know with you know with with
the right label is you know fair game to
it okay in particular you know what
happens is that you know essentially
this is the point about like other
examples being a human phenomenon it's
just about how differently we classify
things and our models do classify things
any particular if you want to have
interpretable models you have to think
about that the training time because you
have to ensure that your model uses the
patterns that you will be able to
understand later okay and that's not
what's happening that's not what's
happening automatically because you know
there's a lot of other helpful
correlations in the data that we
actually as humans don't really use and
maybe cannot use okay so since you need
additional restrictions priors on what
the features you know your mother should
used to make a prediction okay so the
interesting kind of implication of that
it's kind of okay so if I know now that
you know in particular robustness is
about relying on
spurious patterns that maybe are not
kind of what we as humans want to depend
on in our decisions you know how do
models that are robust meaning they
don't rely on this on the robust
patterns you know how do they you know
how do they actually you know how do
they look like and essentially you
realize that one thing about
Smuggler's other than digester he robust
is actually that they tend to be like
for visual models perceptual aligned so
in particular you know that kind of you
know one way of visualizing kind of why
like how the models making prediction is
just to you selling see maps so when I
look at every pixel and I look at the
how much influence that pixel has on the
prediction that I made and if you do too
sometimes models you will get salience
in maps like this which again most of
the time they do the right thing but
clearly also does depend on stuff they
should not be depending okay and that's
what you kind of you get however if you
do the same thing to a robust models you
get patterns like that which are again
much much more aligned to what you as a
human would do so you can view
robustness as a prior that you know for
kind of focusing on new meaningful
features you know in the model and
ignoring some other you know potentially
helpful correlations but very not know
the human aligned correlations and this
actually goes you know even further so
you realize that if you examine the
representations of robust models kind of
the neurons are really corresponding to
some kind of well more interpretable
features you can do cool stuff like once
you realize you know which given
corresponds to stripes to like nice
bright stripes what you can do is you
can morph your image by saying okay can
you give me a version of this image of
has this neuron more excited and then
you will get a striped a striped dog and
also you can do kind of this kind of
very cool like interactive data
exploration where you can start with
some you know some input and then sorry
and then start more thinking into an
input of a different class and you can
kind of do it interactively essentially
you can kind of change the future of the
model to make it kind of look more from
to in class and this is kind of exactly
like you know convincing not only for
the model but also for the human so this
is just an example of 23 class you can
also do a lot of other stuff like you
know you can have you know this gun
paint where you kind of you know you
just like sketch something and then ask
the model to refine it to make it more
photorealistic you know and essentially
you know and also what you can do is you
also can do like input manipulation on
things that were not even trained on you
can kind of hear like make someone smile
change their hair and so on and so on
and all of this is completely fully
data-driven just the key ingredient is
that huge robot fatherless so the
features they use are the features that
you as human use and that's why these
changes are convincing not only to you
but not only to the model but also to
you okay and you can play with the demo
over here it's actually you know pretty
a lot of fun because you can upload
whatever image you want and okay so
there's I told you is that kind of the
Spirit Squad like that these
correlations and the fact that we can
plant them is the problem from security
point of view and the fact that you know
we kind of have them naturally arising
data is also coming from the previous
security but also interpet ability okay
is that the only reason why we should be
worried about this you know correlations
you know Spanish appearing well as I
told you already know that the problem
is that collections can be weird and you
may be the fact that you know the bowtie
correlates with a cat is clearly like
it's clear that we messed up something
but in the end of the day it's like
distinguishing between cats and dogs you
know who cares
however you know when we move to other
domains more high-stakes domains exactly
the same problems appear so there are
now some of the studies the tracking
there was a there was and still is a big
push of using machine learning for
instance for medical imaging
tasks here for like you know recognizing
different you know different you know
different conditions on in x-rays now
people realize that there were some
models that looked like seemed to do
amazing job until they examined that may
be too close and they realized what they
were doing they were essentially looking
for instance like at the type of the
machine that took the x-ray and
essentially extracting collections from
that so if you want to detect
tuberculosis it's a very rare disease in
like developed world so probably like
most of your positive examples will come
from unalaq's know under developed
countries and then they usually have
older machines or like some portable
machines and you can just tell if it's a
positive or negative example by just
guessing you know by just recognizing
which machine took this kind of
this kind of in on this kind of a photo
so again there is this put there is a
correlation that seems helpful but all I
can our model latches on to it but it's
obviously completely undermining the
usefulness of the model and similar
there is like other things with like
taking to Moorea is that essentially
like you know the model learned if there
is you know if there is a measure tape
in the you know in the in the photo or
not because now usually the doctors use
measure tape as a reference point okay
and so you know predictive buttons are
by far not always good and kind of you
know I again like just we need to be
aware of that and we need to kind of
engineer around that and I think that
robust models provide a good way to
exactly like well at least start getting
us on the way to try to you know
understand I could do some
counterfactual analysis to detect you
know what are the real reason why our
models work so here's an example so here
we have a input that correct correct
label is insects but for some reason our
classifier claims this is a dog and at
first it is very confusing
until you actually use the robust
representation you ask okay can you
enhance more the two neurons that
actually made you classify this this
image as dog and when you do that you
will get a morphing like this and then
essentially if you look in this lock top
left corner you realize that actually if
you look back at the drill image that
indeed if you squint your eyes this kind
of looks like a dog and then like no
it's an image net classifier and
actually has you know there's a lot of
dogs in image and justification tasks so
the models happiness - hallucinating
dogs wherever they they can and
essentially that's why kind of it made
the prediction not that it did because
you know it learned that there's a lot
of dogs in the world and it always
intuitively looking first for a dog okay
so kind of you can view robustness is a
framework for starting to control like
which correlations you know will extract
from the data and trying to engineer to
make sure that the bad ones are not
caught up okay let me you know let me
conclude so again I like to say I'm not
sure I have to say to this room but I
like to say it is like know as excited
as we are like we have to realize that
like machine learning is a sharp knife
you know it's just like it's extremely
useful when used right
is very dangerous if used incorrectly
and in some ways this kind of collation
extraction property of malice is exactly
the you know the blade
you know the double-edged blade of this
of this office machine learning is is
the thing that gives us the power that
kind of we you know that's the way in
which they kind of do like that's how
they get advantage over us because they
can find these collisions where we will
be completely lost but that's also
exactly the thing that make leave like
no leave them completely astray and kind
of I view in order door busting in
particular as a kind of framework and I
attempt a tool for trying to correct for
the bad effects of this you know of this
of these features and kind of try to
give us a like not only like like more
you know more secure or like or robust
machine learning but actually a reliable
and more interpretable and select better
machine learning in general okay that's
all I have to say thank you
[Applause]
yes so yeah that's a great question so
yeah kind of when you move in the visual
domain the good the good thing is that
we are very good baseline and we can
kind of have you know we know what is
the right priorities we can tell where
the friar is wrong in other domains in
medical or others it's very unclear if
we have the right pair like most likely
we don't so honestly like again there is
no kind of there's no like no magical
way to do it I think the whole point is
is just to kind of use these tools to
just try to understand what is the prior
and then try to use independent kind of
know maybe then talk to your biologist
friend like try to understand what we
understand about biology to confront in
this if this prior is plausible you know
under you know like under like what we
know for sure from other sources but
yeah you know in some ways the exciting
thing about applying a male in biology
that it can kind of think about biology
in quotes in very different ways that we
do and this not necessarily doesn't mean
to be a bad thing so individual domain
is clearly the bad thing in biology it
might not but we have to be very like
very kind of aware that you know we
might also be fooling ourselves so
essentially like once we have some
discovery based on that which is very
you know very thoroughly vetted to make
sure that you know that we didn't get
fooled so is it again so there this you
know plate becomes both more useful and
also more dangerous there so there is no
there's no magic recipe okay thank you
[Applause]
you