What Machines Shouldn’t Do (Scott Robbins)

right
welcome everyone to uh today's ai tech
agora
um today our speaker is scott robbins uh
before passing the word to him i see
that we have quite a lot of new faces
here
so i will give a very quick introduction
into
[Music]
who we are so i'm arkady i'm a postdoc
at the delft university of technology
and uh
here we have a really nice
multidisciplinary multi-faculty
uh initiative called aitec and uh at
aitec we look
into different aspects of what we call
meaningful human control
so we are focusing uh not only on kind
of general philosophical
uh aspects of it but also on concrete
engineering challenges
postgrade so if you're interested in
this
check out our website
and also subscribe to our twitter and
youtube channel
to stay updated of the future meetings
and
if if nothing else comes up
i'll pass the word to scott
all right thanks a lot and uh thanks a
lot arcadie and
aitech for inviting me i'm pretty
excited to give this talk i'd hoped it
would be in person but
alas we will have to make too um
so while i share my screen
really fast um
can everybody see that now
can somebody let me know if you see that
yeah all right sounds good
all right thank you um so a little bit
about me i'm just finishing up my phd
at the ethics of philosophy and
technology section at
to delft at the tpm faculty
i'm writing on machine learning and
counterterrorism and the
and the ethics and efficacy of that and
this is a paper that didn't make it into
the thesis that i just submitted on
monday
but it's something that i i think the
thesis kind of leaves on set and kind of
leaves the future work that i'm really
hoping to
get out there as soon as possible and
right now i'm titling it what machines
shouldn't do a necessary condition for
meaningful human control
all right so before i say anything about
what machines shouldn't be doing
i i i have to clarify what i mean by
what a machine
does and when a machine is doing
something or more specifically when we
have delegated
a process or a decision to a machine
and what i mean by this and for the
purposes of this paper in this
presentation
is that a machine has done something or
we have delegated a decision to a
machine
when a machine can reach an output
through considerations
and the waiting of those consideration
considerations not
explicitly given by humans so
the point of me doing this and to making
this clarification
is that i want to distinguish between
you know more classical ai
like good old-fashioned artificial
intelligence symbolic
ai or expert systems and things like
that
from machine learning or contemporary ai
and in the classical good old-fashioned
ai the considerations that we
um that go into making a particular
generating a particular output are
explicitly put there by human beings
it may be extremely complicated and may
look um it may
still simulate our some way
and we may believe it's doing more than
it is but really there's there's human
beings behind that
that path that it's that it's following
to generate the output
this is in contrast to machine learning
or many methodologies and machine
learning
where the specific considerations that
go into generating an
output are unknown to the to any human
beings but even the human beings that
programmed it
so that's the case in the in in much of
the hype surrounding ai today and many
of the successes that we've seen in the
media
like alphago beating the world go
champion
or chess algorithms or many of the other
other algorithms out there
so we've delegated a decision to a
machine or
in this example here the move in the
game of go
to a machine because that machine is
actually has its own considerations
loosely speaking
for how it generates the output
so this we've delegated in in the way
that i'm talking about it we've
delegated many
decisions to machines everything from
detecting whether somebody's suspicious
or not
to predicting crime to driving our cars
diagnosing patients
and sentencing people to prison
um going even more fraud detection
facial recognition object recognition
and even choosing our outfits for us and
developing this presentation i saw
a lot of really random algorithms out
there or
applications out there and so with all
these applications
i think some of them kind of freak us
out and i think
autonomous weapon systems are some of
the ones that really come to mind
is the most classic example of something
that comes to mind that
that scares us about you know like
should we be really
doing this within with an algorithm or
how can we do this responsibly
and it's fueled this huge explosion in
ai ethics
i think a justified explosion in ai
ethics because i think there's something
novel happening here
this is the first time we're really
delegating these processes to machines
not just automating the processes we've
already determined but actually
delegating
the the act of choosing these
considerations to machines
and so now we're worried about the
control we have over those machines
and specifically i think that
all of these ai ethics principle lists
and a lot of the work on ai ethics
whether it's explicit made explicit or
not is really talking about
or adding to in some way trying to
realize meaningful human control
and so before i get to the specifics of
what i want to add to meaningful human
control
i wanted to say a little bit about some
of the proposals that are out there
already and a little bit about how i
classify them
i have made a distinction between
technology-centered meaningful human
control
and human-centered meaningful human
control and what i mean by that
distinction
is is what i'm trying to capture here is
where people
are putting the spotlight or putting
their focus
on realizing meaningful human control so
if it's technology centered then we're
really thinking about the technology
itself
what are the design requirements of the
technology or what can we
you know add to the technology so that
we are better equipped
to realize meaningful human control
whereas in the human centered approaches
it's more about where can we place the
human and what capacities or
capabilities does the human being need
in order to realize meaningful human
control
so starting with a technology centered
meaningful human control
there's a few proposals out there that i
consider to be the biggest ones
and i don't mean to say that these are
all necessarily put out there
explicitly to realize meaningful human
control it's not like these
papers are all saying you know this is a
way a proposal for meaningful human
control
but i've argued in the past that some of
these are are
indeed doing that or that would be the
moral problem that they're trying to
solve if they're solving one
so first is explicability and
explainable ai
which is the idea that if we can make um
an algorithm output not only you know
its
you know its output but also give us
some kind of idea of how it came to that
output
in terms of considerations that went
into that output
you know this could perhaps allow us to
say well that was a bad
uh that output should be rejected
because it was based on race or gender
or something like that
something we considered to be um not an
acceptable the way
a way to make a particular decision um
i've written a paper on this
this proposal and i'm
not too thrilled with it i think that
it's a good idea
but it it fails and there's there's
still a good idea to
try to make ai explainable i still think
there's reasons to do it
but it doesn't solve the moral problem
that it attempts to solve
if it's if that's what it's doing and i
can talk more about that or i can direct
you towards the paper
about that in the question and answer
period
then we move on to machine ethics which
i i think is not necessarily a proposal
for
realizing meaningful human control but
it is a way to say that
that it's saying that if we can endow
machines with
moral reasoning capabilities and allow
them to pick out morally salient
features
and adjust and be responsive to those
features then we don't need
humans to be in control anymore really
the machines and
robots and algorithms are controlling
themselves
with these you know ethical capabilities
i couldn't be more negative about this
approach and i think some of that has to
do with the reasons that i'm going to
get into later on
but i've also written a paper about this
with amy van weinsberg
where we argue that there's no good
reason to do this every reason put
forward
fails for either empirical or conceptual
reasons
and again i can talk more about that and
hopefully it becomes more clear
throughout this presentation
at least one of the reasons why this is
a bad idea
and then we get to track and trace which
is a proposal put forward by
philippo santoni de sio and jeroen
vanden home and hoven here at
uh tu delft and tpm in particular and
they they have actually a really nice
paper i highly recommend
people read it if they're interested in
meaningful human control
um philosophically you know it has some
depth to it
and they put forward two conditions
that we need to meet in order to realize
meaningful human control
the first is a tracking condition which
is about
the machine being able to be responsive
to human moral reasons
for for their outputs so such that if
a morally salient feature pops up in a
context that would cause a human
to change their decision or output then
it should also cause the
algorithm to change its output and the
second is a tracing condition
which states that we should be able to
trace responsibility back to a human
being or set of human beings
such that that human being should
knowingly accept moral responsibility
and should be ready to accept
the moral and
moral responsibility and accountability
for the outcomes and outputs of the
machine
all right moving to human-centered
meaningful human control this is the
classic on the loop in the loop stuff
um i think uh and this is again about
the human where is the human
in this process and what capabilities do
they have and an
on the loop the human is is kind of
overseeing
what the algorithm is doing so that the
human can intervene
if necessary it's to prevent something
bad from happening
and i think a good example of this is
tesla teslas which
kind of stipulate that the human has to
be you know have their hands on the
wheel
and be ready to take over at any time if
something bad happens
and really you know all moral
responsibility is
is with the human i think this is more
of a way to protect
their company from lawsuits than
anything else it doesn't seem to be a
very good
way of realizing of having any
meaningful human control
of of the machine as it kind of flies in
the face of human psychology and
human capabilities to remain aware of
their surroundings
despite an automated process which works
most of the time there's some
interesting work on that
i think even from tu delft so i don't
think it's it's meaningful human control
but it's it's an attempt at
um i i think that's what they're trying
to do it's just not
working very well and then in the loop
is a little bit stronger than on the
loop and that it requires a human to
actually endorse
or reject outputs of the machine before
the consequences happen
so you're not just in an overseeing role
anymore you're
you're actually a part of the process
again this kind of suffers from
you know flies in the face of human
psychology and that you know we suffer
from many biases like automation bias
assimilation bias and confirmation bias
which is going to make this incredibly
hard to be
meaningful control even though i think
it could be said we are
in control
um and what i really what i really want
to say with all these is not that i want
to
and the reason i don't go into specifics
on my attacks and all of these positions
is that i don't think it matters too
much for the purposes of this paper
the real point that i want to make here
is that even if some of these
uh solutions or proposals will play
indeed play a part and i think some of
some aspects of them at least will play
a part in meaningful human control
they've they're kind of working this all
out
after we've already made a huge mistake
um when we've made a mistake
and we've already hit this iceberg and
now we're just rearranging the
technologies
and the people in the socio-technical
system
and hoping that everything will work out
but it doesn't matter the ship's already
sinking
and specifically i think the mistake
that gets made
is that we've delegated a decision to a
machine that that machine should not
have been delegated
and as soon as we've done that no amount
amount of technical wizardry
or organization of the human and the
technical system
is going to fix that problem we've
already lost control
meaningful control over these algorithms
so that's what we have to figure out
first and that's what i plan to try to
do
in the next half of this presentation
so to jump to my conclusion that i will
then defend
is that machines should not have
evaluative outputs
and specifically machines should not be
delegated evaluative decisions
and i i consider a value of outputs or
evaluative outputs are
things like criminal suspicious
beautiful propaganda
fake news anything with bad good wrong
or right
built into the labels or the out or the
outputs
so when we say somebody's suspicious
we're not just saying that somebody
is is standing around in one spot for a
while with a ski mask on
we're not just saying oh that's
interesting we're saying it's bad
there's something bad about what's going
on we're
we're loading a value into it and we
describe somebody as beautiful
we're not just saying that they have a
particular outfit on or they look a
certain way
it's just in a neutral manner we're
actually saying something good or
something about the way people ought to
look
and same with propaganda we're not just
saying that there's a picture picture
with a political message
we're saying that there's something bad
about this picture with a political
message
it's not the way things should be and so
any of these outputs that have values
built into them
should not be delegated to machines
and i'm not just shadow boxing here i
think most of us here will know this
that these types of outputs have been
delegated to machines quite frequently
there's no shortage of proposals to do
this
[Music]
for these are four examples here like
detecting propaganda
and here i do want to make a little note
that remind us that
in the beginning i was talking about
what i mean by a machine doing things
and sometimes algorithms are delegated
the task
of flagging propaganda but that doesn't
always mean
um that it's doing something in the way
that i've talked about it doing
for instance europol flags propaganda
but they do it based on
a hash that's generated by
videos and pictures that have already
been determined by human beings
to be propaganda and the idea then is
just is there a new post or a new
picture a new video
that is the exact same as the one that
was already posted and taken down
that's not a machine that's just
automating um a process we've already
done the value judgment
ourselves and then we're just delegating
the task of finding
things that that match those value
judgments
in this case i'm actually talking about
a machine
or an algorithm that detects propaganda
and novel propaganda on its own
and then moving on to you know we've had
algorithms that detect
criminals just based on their faces or
purport to do so um ai that detects
beauty ai that detects fake news
we have one of my least favorite
companies hirevue
which works for which is used by many
fortune 500 companies now
which purports to be able to say whether
a candidate is okay good great and
fantastic
or and probably bad as well
and and then i think that one of the
more infamous versions which is the
chinese social credit system which from
my reading is trying to say what is a
good
citizen so
remember these are all things that i'm
saying algorithms shouldn't
be delegated and then the question
becomes why
why why can't we delegate those things
and i want to put forward
two arguments here one has to do with
efficacy roughly that uh i'm going to
argue that we
we can't say anything about the efficacy
of these algorithms and if
and if we can't say anything about its
efficacy then we can't justify its use
and that any use of it is uh is out of
our control we've lost control at that
point
and then the second argument is more of
an ethical argument about
a more fundamental loss of control um
over
uh of meaningful human control over a
process and i will get more into that
when i when i get to that section
so first i want to say that efficacy is
unknown in principle for evaluative
outputs
and that every each evaluative output is
unverifiable for example
suspicious and in this example in this
picture here this
man is being labeled as suspicious and
in this this judgment this output here
and if we wanted to say
well did the algorithm get this right
was the algorithm correct and labeling
this person suspicious
we in principle can't do that because
you might say you might argue
wait we can if the person stole
something well then they were indeed
suspicious
but that's not what suspicious means
suspicious can be
somebody can be suspicious without doing
anything wrong
and somebody can do something wrong or
steal something without having been
suspicious it's called being a good
thief
so if we can't evaluate whether the
algorithm was correct on this one
instance
and i'm arguing that it can't on any
specific instance then we can't say if
this algorithm works at not or not at
all
there's nothing we can say about the
efficacy of this algorithm and therefore
we're kind of out of control we've lost
control we're just using a tool
that's as good as a magic eight ball at
that point
and this is opposed to an example like
an algorithm that's supposed to detect
weapons
in a baggage and baggage at the airport
in this instance if an algorithm says
classifies this
bag as having a weapon in it or more
specifically having a gun in it
we could probably we probably wouldn't
just arrest the person on
on just the fact that the algorithm made
such a labeling we would
look into the bag and find out does it
indeed have a weapon or a gun in it
and if it does then we know the
algorithm got it right at that point
and we can say something about the
effectiveness of the algorithm because
we can test this on many bags on many
examples and determine how good it is
at in detecting weapons and then we can
have some place
in justifying its use we can say it does
indeed get it wrong
sometimes but we have a process to
handle that because
we have enough information to remain in
control of this algorithm
um the other thing i want to say about
efficacy and and uses
using this suspicious label again is
that
the context change over time so one year
ago if this kid would have come into a
store that i was
in and not at night i would have
justifiably been
worried and probably thought this person
was suspicious but
i think uh in the context of a global
pandemic right now in the netherlands if
i saw
the same person wearing a mask in the
in a store that i was in i'd probably be
thanking them for actually wearing a
mask
as i see so little of here in the
netherlands despite the spiking
corona cases so these contacts change
and that's that's something that
algorithms are not good at
they're good at fixed or machine
learning algorithms specifically they're
good at fixed
targets something that they can get
closer and closer to the truth with
over more data but value judgments are
not some
are not those kinds of things so we have
to be so even if we could
solve the problem that i just outlined
which we can't tell whether it was
we can't verify any particular decision
that we did solve that problem well then
we'd also have to worry about
the context changing to make those
considerations
um and make those considerations change
that ground that judgment
all right now moving into the ethics
part of the argument where i'm arguing
for a more fundamental lack of control
what i mean by fundamental here and it's
different from the control that
i i see so often talked about like in
the autonomous weapons debate
where we're really thinking like okay we
have this algorithm out there and it's
targeting people
how do we make sure that there's still a
human being around
that can make sure like can take
responsibility or have control over that
process
what i mean with this fundamental part
is that the control over choosing the
considerations that ground our value
judgments
so the actual process of deciding how
the world ought to be
in any of these contexts that is a
process that we have to remain control
over
in control over it doesn't make sense to
delegate that
to anybody but ourselves we human
a machine is not going to be able to
just decide how the world should be
and then we all change just because of
an output of an algorithm that doesn't
make any sense
we have to decide how the world should
be and then create algorithms
to help bring us there so in the example
of
you know going over cvs to find a new
candidate for a job
say in academia we might say that
there's certain considerations that are
important for that
for instance the number of publications
um their reference letters
you know what they did their
dissertation on there'd be all these
considerations that go into
such an evaluation and we usually have a
committee
to decide like how do we want to
evaluate candidates for this particular
position because it may not be the same
considerations every time
and that conversation that we're having
not only as a small committee or as an
individual
but as a society to say what is
like a good what is a good academic and
then from there determining the
considerations that will lead
us to choosing those good academics or
bad academics or good candidates or bad
candidates
that is our process that's what we need
to remain in control over
so delegating this when we delegated an
evaluative output to
a machine we are effectively seeding
that more fundamental level of control
and then continuing on with this theme
even though we're having a conversation
today about what a good academic is or
we did that you know we were doing that
50 years ago
now things have changed drastically in
the last
you know bunch of decades or even in the
last 10 years i think
even at tu delft you can see that
certain
characteristics or considerations are
used that weren't used before
about what a good academic is for
instance valorization is much more
important
than it used to be we've decided that
that is part of what goes into making a
good academic
or maybe teaching is a little bit less
important or maybe it's not the number
of publications anymore
but the quality of their top
publications
this is the conversation that we have
it's also a conversation that changes
as we learn new information or as the
context around us changes
it's not that the value changes itself
but how we ground those values with
considerations changes
so that is that should be left up to us
to do
next is the is this
what i'm calling a vicious machine
feedback loop and it's it's a concern
that i have
that by delegating this evaluation
this evaluating process to machines that
we could be
influenced by these machines on how we
end up seeing
values and grounding values so the
process starts with us
human beings building these algorithms
training it
you know labeling the data however that
process works we
are we are doing that in our way and it
will always be biased it may not be
biased in a bad way
but it's still biased even if it's just
to the time that we live in
and then that feeds into an algorithm
which goes into an evaluative decision
which then spits out these these
evaluations to us and over time what i'm
worried about is that
we could be influenced by how the
algorithm makes decisions
we could start seeing candidates for
jobs differently because we
we've seen so many evaluations of an
algorithm
show us what a good candidate is and
then we start taking those things on
and this isn't an entirely new problem
um it's happened before with other
technologies
like even the media we're constantly
worried about
feeding in the idea of who is beautiful
and
body shape and body image to children
for instance or barbies or something
like that
all this stuff feeds into their idea of
what beautiful is
and affects them later and we usually
talk about this in a negative way
we don't like that that they're feeding
in a specific body shape as the only way
to be beautiful
and then that actually affects how they
see it which affects their
their way of trying to realize it and
who they think is beautiful well
those same evaluations are now being
delegated to ai we don't even know what
considerations they're using
and i don't think not knowing is better
than knowing and it's bad
they're both not a good effect on our
evaluation our ability to evaluate
the final thing i want to say about this
this this ethical control argument is
that
people's behavior will adapt to
evaluations part of the reason that we
perform evaluations
is to get people to um we're saying how
the world ought to be
if we're saying somebody did a great job
we're saying well if you want to do a
great job too you should
do it more like this or you should look
more like this to be
beautiful or that's what evaluations do
now um of course that will change
people's behavior
and in the case of ai we've seen some of
these overt changes where people figure
out that ai is doing something
like these students who figured out that
their tests were graded by ai
and we're actually able to achieve
hundreds rather easily
um that affects their behavior and of
course in this situation well that's not
behavior that we want to we don't want
to change their behavior that way
that evaluation is failing that's partly
because
we haven't had the conversation we're
delegating the conversation about what
it is we want
what behavior is good in taking a test
to a machine
and that doesn't based on what i said
before that doesn't make any sense
that's the whole thing
we have to determine what is a good
what is good for a test and then we can
have ai help us
evalua help us evaluate it by picking
out
you know descriptive features of it but
it can actually
do that process for us it can't pick out
the considerations
that make a good test or a good
candidate or any of these things
that is us losing control over a process
that is fundamentally
our process
um so then i started this you know the
whole thing is
called what machines shouldn't do so i
wanted to reiterate you know i think
some of it should be clear by now
but um what i think machines shouldn't
do based on the arguments that i've used
and the first is um they're all
evaluative outputs but
aesthetic outputs so judging what is
beautiful
or what is good you know what's a good
movie what's a what's a bad movie what's
a good song
we have you know algorithms that are
trying to trying to say what the next
good movie is going to be i think these
these don't make any sense based on what
i've said not only can we not check if
the algorithm works or not
but we can't um but we're losing control
of that process of having that
conversation about
what is good um regard to the aesthetics
the same with the ethical and moral
you know we shouldn't be delegating the
process of evaluating
um of coming up with the considerations
that evaluate candidates or citizens or
anything like that
to machines that is our process and
finally
i add this last one in i don't think
it'll make the paper because i think
it's a separate paper
but there's been so much talk of ai
emotion detection
that i wanted to mention it because i
think it fails some of the same thing in
the same way
that some of the aesthetic and ethical
or moral failures go
and that is we emotions are not
verifiable in the way that the gun in
the bag was
and furthermore there doesn't seem to be
based on the science that i've read
um the anything more than pseudoscience
grounding the idea
that we can use ai to detect emotions
all right so now i'm getting to the
conclusion now
i think this is obvious but
unfortunately based on all the examples
i've shown
i think it still needs to be said ai is
not a silver bullet
it's not going to evaluate better than
we can and it's not going to tell us how
the world should be
it can help us realize the world we've
determined
the way that it should be but it's not
going to be able to do this this is
fundamentally a human conversation
a human process that we need to we need
to keep going and it will never stop
and then have the technologies around us
help us realize those dreams
and then lastly i want to keep
artificial intelligence boring
i know it's not as exciting as being
able to figure out
ethics you know determine exactly what a
good person is and then we just go
toward
we just follow the machine that would be
really exciting if we could do that
but it's just not something that can be
done
but i also don't want to be completely
negative here i think despite
all of the the last you know half an
hour or so
i'm actually really positive about many
of the benefits that artificial
intelligence
and machine learning could bring us i
just think that it needs to be more
focused into the things that
it's possible to achieve and that is
identifying the descriptive features
labeling those descriptive features
that ground our evaluations
so instead of determining who is
dangerous it can determine
a gun in a bag instead of determining
who a good candidate is
it can it can rank the the
cvs based on number and a number of
publications
you know it's hard to read 100 cvs but
an and our
machine learning algorithm could
probably do it really fast and we could
verify that it was doing it correctly
this is still hugely beneficial and i
think we underestimate
the power that that could be so i'm
going to leave it there
i really appreciate you guys listening
to me for the last half hour and i
look forward to your questions um
i'm going to stop sharing my screen in
like a minute just so i can
see everybody i feel quite alone when
i'm looking at it like this
so uh but thank you very much
thank you scott uh that was very
insightful and uh we actually have a few
questions already in the chat
so while i'm reading those uh i would
appreciate if
people uh write their question or at
least the indication that they
have a question in the chat because
there are already quite a few of them so
if you raise a hand i might as well miss
it
um so i'll start uh with the first
question
that arrived i think 30 seconds after
you started the attack
uh so that was uh from enough so
you know would you like to ask your
first question
yes thanks scott for your presentation
very nice
i i was i got stuck when you uh started
to make the i started to explain the
difference between automated
and delegated and yeah i was worried
about it as i gave you as i wrote in the
chat if you if you look take the example
of face recognition
right um then there are face recognition
systems that make use of explicit
explicit selected facial features like
you know position of your nose
distance between the eyes color of the
eyes etc etc and there are some that
learn
these features and we do not know
necessarily what kind of features that
are
both of course have the same effect
namely they recognize people they
classify people etc etc
but one you would call automated because
we as humans select the features and the
others delegated and i don't get that
why it is even important to make that
distinction
uh well i think it's fundamentally
important because we
in one case we're deciding what features
are important
for grounding and a decision in the end
and so we have it's the control problem
right we have that control
but if we are delegating that to a
machine which is fine in facial
recognition i don't
i have lots of problems with facial
recognition but not for this reason
is that we algorithms
machine learning algorithms specifically
are able to
um use its own considerations and it
becomes more powerful and the in the way
that i
mean by that is that it's not confined
to
what we can think of or human
articulable reasons
it actually has a host of other reasons
that we could never understand
i think that's what gives it its power
but it does matter because one is going
to be explainable and one's not
so what is that the word then so i was
looking for that word explainability so
you're you're
the the distinction between uh automated
and delegated is whether or not
it's explainable i think that would be
i i just so it'd just be a clarity issue
then then i
i think that works fine for me but i
would just i prefer the
delegated the decision because i think
that makes it more clear maybe for my
community
but i'll leave it today now you gave a
very if i may you gave a very nice
example of a decision tree with two
levels in the tree right yeah realistic
decision trees have
thousands of levels of course yeah yeah
is that explainable yeah so you see i i
mean it's fundamentally explainable
between the two huh yeah but it's
fundamentally explainable right
whether you have a thousand levels or
two levels you can still explain it
i mean somebody could explain it in the
end we could in principle get an
explanation but with machine learning we
can't
right now at least in principle get an
explanation so there's a difference
between the two and
i'm very interested about this because
uh you mentioned this one example the uh
ai that was used to determine whether
someone should be hired or not or the
quality of
candidates right and i think you made
really good points against that
uh being used but um when we are
when you were talking and when you are
talking about this
um i'm wondering would you say it's
perfectly fine to use an ai
if it's a gopher good old-fashioned ai
with a decision tree behind it
and only problematic when we use machine
learning
uh right so i think it is so there might
be other problems associated with
gophi in that situation but i think we
still have i mean the way or the
arguments that i've made today
we still have control over what
considerations are grounding the
evaluation
so for example if we're using a hiring
algorithm that's basically just saying
you know how many if it's in the
academic sphere is there
um do they have greater than five
publications
um and did they get a phd that has the
word ethics in it
and those are the only two things we
care about separating out the things
well of course we can we can argue about
whether that's good those are
considerations are good
but it's still us deciding what
considerations there are so it doesn't
fall
foul to what i've talked about today it
still may be problematic
but um not for this reason
okay so uh if we move on to the next
question
um so providing a link to the paper
somebody posted it already
so uh then we had uh a question from
reynold about
having a disease uh would that be an
evaluative output
you know can we clarify what the
question was about
oh yeah i think the question is pretty
clear huh yeah
pretty clear don't worry um so i i think
i think i use the example in my paper
sometimes about
detecting skin cancer and an algorithm
there's algorithm that can uh
look at a mole and take that as an input
and evaluate whether you have
uh skin cancer or not so this i don't
think is evaluative because it's
it's verifiable right we can just take a
biopsy and say well did the algorithm
get it right or not
it's a zero or one do you have skin
cancer or not so in that case it's not
an evaluative output no there's no
there's no evaluation i mean
i guess in the uh if it was more mental
if we consider mental issues that's the
next line in the question exactly
well then it's uh i think there's a
blurry line there but
uh i i'd not experienced enough in the
in that field to know whether so i have
a very strong feeling that and i think i
saw that somewhere in the chat window as
well that that
what what you're worried about is
whether or not there's a ground truth
an objectively verifiable ground truth
i think that is yeah i mean i we could
say that
i'm happy with that language okay
good thanks good now we have a question
from david
uh he is wondering whether you can
acquire equate
evaluative outputs with outputs of moral
evaluation
is there necessarily a moral component
or maybe you can clarify that
yeah thanks thanks for the talk also uh
scott
but uh i think enol
just touched upon it in the previous
talk so
i was wondering what you mean with
evaluation
and i think the ground truth part that's
probably
more general than moral evaluation but i
had the feeling you were thinking about
moral
evaluation but also like i said in that
in the talk it's also a static
evaluation you know anything's with with
value
attached to it but i think that does
cause i think the verify ability here is
very important
and it may indeed be easier just to say
is it um is there this ground truth and
and verifiability
part of part of it i think that's that
could be possible i need to think more
about it
okay thanks okay uh now we have another
question from another
also drilling down on the no no no let's
take someone else come on okay
yeah we'll skip this one so we have a
question from jared then
uh about uh the kind of
the gray area between evaluative outputs
and uh something more objective
jared do you want to ask a question um
yeah so thank you very interesting
presentation uh
and so one thing i was thinking about is
that it would be terrifically easy for
pretty much all examples you gave
to reframe them uh as if they are not
giving evaluative outputs but more like
objective judgment
so you can say the ai does not evaluate
appearance it judges similarity to other
applicants based
on or as ranked on suitability and you
can
sort of reframe what your
ai does to something objective and then
say
it's just meant to inform people the
people make the decisions and we just
do this objective bit that informs the
decision
uh and i have a feeling that
focusing on my outputs um might not be
the right
emphasis but i'm not completely sure and
i just wanted to throw it right see
what's uh
and and i'm worried about that too i'm
intentionally making this
bold so that i can get pushback but
i think what what you just said is is um
i like that better in terms of we're
just um
finding applicants that are like the
applicants
that are the people that we've hired in
the past that we've classified as good
so we've done that process of judging
whether they're good
however what considerations are
grounding that similarity part we don't
know
how it's reaching that similarity so i
think it still falls foul
to that and what i think and um in my
more hopeful parts about ai
instead of trying to find you know what
applicants are good based on the
applicants of the past that we've
determined as good
we should figure out well maybe we
should think about what makes them good
what what is it about them that makes
them good and then when we determine
that
then we can use ai perhaps we need ai to
automate
being able to find that feature and to
be able to sort those
applications that's what i really want
because you can't say
um because this suitability thing i
think it's a nice work around and i'm
sure the technology companies will do it
because it makes it uh makes it sound
better and makes
it it forces more of the responsibility
to human beings but i don't think it's a
justified or
meaningful control then i mean what's
the difference between more suitable
and like our better applicants from the
past and just a good candidate
it's basically amounts to the same exact
thing and then it falls into the same
problems
thank you okay next we have a question
from sylvia
sylvia can you plug in and ask the
picture
yes yes um if i can also just add
something about what
what um uh jared was saying then
i think it would at least be an
improvement then
it's clarified what the ai actually does
so for example matching
similar cv because in terms of social
technical
system at least you remove the um
the the tendency to uh trust that then
the ai is gonna have this
additional capability respect to us and
you know we should trust then what the
computer is saying
maybe i agree i think it's a step
forward like that it's definitely a big
step forward it's it doesn't it still
falls victim to someone what i've said
but it's a step forward i agree
yeah but now what i wanted to ask you is
i mean i
i actually don't mind your idea of
saying okay let's keep it boring
because to me just sounds like that just
keep it within the boundaries of what it
actually
could do because it can't do this
contextual
kind of evaluations but it doesn't it
just boils down then
to it can verify very well
tangible things like objects like your
organic sample or the
the checking the moles and
possible skin cancer against then
intangible things so is it is it boiling
down to let's just keep it to
objects or things well
i mean i i think much hinges on the
verifiability
i mean like i guess i mean
earthquakes are tangible you know like
we could predict earthquakes with an
algorithm
and we would only know after the fact if
it got it right but
i you might be right it might be just
tangible it depends on what we mean by
tangible um
possible but really i think the key word
is verifiability
is is can we can we verify it after the
fact or
during and i think that will matter when
we can verify it but it
it if we can't verify it at all which
i'm trying to say that all evaluative
judgments cannot be verified
so we can't do that and i'm trying to
give an easier way
and maybe verifiability just is easier
and pragmatically i should just be using
that
i'm open to that and i will be thinking
about that more but
um yeah so i will
then can i just leave you with like a
sort of like devil's advocate
for location because then it's the
question that i would ask myself
which is um we can't verify an
evaluation from a human either
if the judge decides that that was or a
jury if you're in a in a common law
system
then that that was suspicious you're
gonna fall into the same problem so
someone that really wants to use the
algorithm might say
but then let's let's make it you know
um statistically sound or whatever like
the
lombroso nightmare that was with the
criminal behavior ai and and
so whether it's a human or an ai we
still can't verify that evaluation so
let's just switch to
let's just you know save money and use
the ai anyway
that would be my biggest problem then
right
and i think you're absolutely right we
can't um we can't
i'm saying evaluative judgments aren't
verifiable so it's not going to be
verifiable if a human does it either
but i think um and if i can go back to
using the phrase
saying how the world ought to be and how
people ought to be
that is up to us to do and the idea that
we're going to use a tool
to do that instead doesn't make any
sense especially if we can't
we can't say anything about its efficacy
so um
if i'm right and i think you know i i
find it hard to create an
opposing argument to say well no
actually it doesn't matter
um how we come to the decision about how
the world ought to be
that doesn't matter we just need to
accept you know we just need to accept
one so that it's easier for the machine
to get there
i don't think that just doesn't make any
sense to me but
um it's something that needs to be
worked out more but i think there's
definitely a difference between a human
not being able to
making an evaluation and us not being
able to evaluate it
versus a machine making an evaluation
and i think one of the big differences
is that a human can explain themselves
about how they
came to that decision and what
considerations they used and we can have
the disagreement about
whether the considerations they used
were indeed okay and of course we can
get into the idea that humans can be
deceptive you know they're going to lie
about how they came to the decision
they were very biased against a
particular person but they're not going
to say that they're going to try to use
objective means
but you know the responsibility falls on
them then you know we shouldn't be
delegating that process to a machine
okay uh herman do you want to ask your
question
yeah great um
uh thanks scott i really liked your
hearing your paper
as but i also have a question on
something that has been touched on be
quite a few times already so the
verifiability of an evaluative judgment
so i still struggle with understanding
what you what you mean exactly with that
so do you want to say that nothing is
suspicious nothing is wrong
nothing is beautiful because if if
some things are beautiful then we can
just see
check whether the output corresponds to
reality
um so your answer you just
gave you seemed to hint that we went to
verify the reasoning
behind the judgments uh so and
that seems to be something different so
is that is it the last thing that you're
interested in that
the the reasoning should be very
valuable or is the uh is it the outputs
that is
well i i think both first of all it's
the output and so
i i think with algorithms they're
they're in computer science it's zeros
and ones
and i think we should be thinking about
um we should be working towards and i
know all the bayesian stuff but
i'm not going to get into that but i
think first at least even if even if i
accept that there might be a possibility
that we could uh verify what's beautiful
or not
which i i think there's a lot of
complications in that because it's going
to be culturally specific
even even person-specific not that
there's no somewhat of
even if it's a human-constructed truth
over it there may be
you might be right then i think i that's
why i added in all those reasons about
context changing our considerations
grounding these judgments are changing
movies that were amazing 50 years ago
if the same movie came out today we'd be
like well it's kind of tired
it's not something that we're interested
that's not beautiful anymore
and it's because the context has changed
we've already heard the beatles you know
we can't have a new beatles out in
anymore somebody has to evolve and and
these
and even with how the world ought to be
the context changes
the climate changes global pandemic how
we
how we live our lives now the fact that
we're doing this digitally
instead these considerations change and
so now that there can't be any truth at
the moment right now
but that algorithms are not good about
this
shifting context in this shifting
situation about how we ground our
judgments what what what a mountain
looks like
overall is pretty much static right i
mean there's some differences in
different mountains
but overall an algorithm trained on
trying to find spot mountains
is going to get closer and closer to
being able to be better and better at it
but that's not the case with evaluative
judgments so
you might be right we might be able to
just say well look everybody agrees
that's beautiful the algorithm got it
right
but then we have to worry about this
changing context
and that's if we can get everybody to
agree on those things which i doubt is
going to happen
yeah so of course there are many
uh metastatic meta
normative views that yeah contradict
some of the things that that you just
said so maybe it could be very explicit
that you're endorsing a
specific but i actually i'd be curious
to know i'd be curious to
to hear if um any because i i
i do read a little bit of meta ethics
especially not meta aesthetics
but even if like for instance real and
moral realism is true
and that there are mind independent
moral truths out there
um i don't see unless unless we can
access them which
even which i have not the moral
epistemology behind that is
exceptionally difficult
and there's no solution to it yet but if
we can't access them at this point
and then the algorithm will say people
have hubris
they believe the algorithm can actually
access them this algorithm is actually
better than us
it can actually detect the real moral
truths well then we'd be left in a
situation where we have to just
trust that's the case we'd have to say
well yeah i guess we
we should kill kill um kill five to say
uh
kill five instead of the killing the one
and the trolley problem i didn't know
that was true but the algorithm said
it's true
it's like why how would we ever trust
that it doesn't even it so even if the
meta ethics
were such that there were moral truths i
don't see that solving the problem
so i'm not sure that a mind is dependent
on a med ethical
viewpoint at this point but i think it's
really interesting and i think we should
talk about that more
let's do that
okay next in line is madeline but when
do you think your question
uh hasn't been answered before because
this goes into
uh context for evaluator judgments
yeah so this is funny the question has
just become more complicated i think
so maybe i just pushed a little bit to
say um
okay so i'm just wondering about whether
or not if
this output of a verb evaluative
statement
is evaluative at all so if the
ai this program says this is beautiful
um that what it's really doing is saying
i've processed this data and here's what
you people seem to think is beautiful in
this particular moment in this
particular context so
it seems like the uh if you're i'm not
so strong on the
technical side here but it seems like if
you're using machine learning this is a
learning process and would be able to
adapt to a certain context
um so i'm just wondering if this
and just i guess i'm just challenging
the idea that an ai
program could come up with the
evaluative statement at all
um well i think you're right i think i
should clarify that more that i mean
yeah machines don't have any intentions
they don't they don't have uh
any you know viewpoints or anything like
that they're just doing the statistical
analysis
but practically when people use higher
view the algorithm
they're getting an evaluative output and
no matter how
you know you could try to put a
disclaimer in there consistently
i'm saying like this is not a real
evaluation this is simply
and put i have in my paper on
explainability
this great explanation of why
alphago made a particular move in the
game of go or how it comes to a
particular move
and really this is what the algorithm's
doing of course it's like
really over anybody's head that's not in
machine learning
and i think the practical situation is
that people are using these algorithms
and taking it as an evaluative judgment
and if we're not going to
and i think language is really important
here and that's why i said we're going
in a step forward if we're saying that
this algorithm has determined that this
candidate is similar to
these three candidates that were
considered good by you before
that that's going a step in the right
direction for sure
i still don't like it but it's it's it's
more accurate to say it like that even
if it's not as exciting to say it like
that
so the concern is about then the
practical use and how it's taken up
by people
um no i think i think that's also a
concern for sure
but i don't think it i don't really
think it solves the problem to just
stipulate that
the algorithm is coming up with uh is
coming up with this
this person is the most similar to these
three candidates that you've had before
similar how i mean that how is so
important
you can't just disregard the how it's
how they're similar
they're similar because they're white
males i mean that's what amazon's
algorithm did
that how was fundamentally important so
i think we can't i think it's just this
this nice clever little trick that some
companies are doing to say like oh well
we just we can just make this little
move with language and change the whole
game
no the same problems exist in my opinion
peace
all right i think i'm next um
this is still enough time okay
can you hear me yeah sorry i was
neutered i was talking all this time
yeah i said that we have three minutes
and the next line is rule so uh go on
and uh it would be great if we could
also get the catalan's question
okay well hi everyone great to be here
uh scott thanks for your talk i really
appreciated it
i think what you're one thing you're
doing is like taking the focus off of
the technology of the you know the logic
etc and giving arguments to uh
motivate us to look at the practice and
all the choices we're making and that's
where i've i've
spent a lot of my my time in the last
couple years as well
um and um like i have a specific
question about
delegation um yeah
where you where you draw the line but
before i say that like i
want to point to one thing i did myself
was
to start looking at all the different
choices that people make
and a lot of what you're talking about
is like how do you how do you determine
what a label is and what can you capture
in the label and is that thing that
you're capturing is that like an
epistemologically sound thing that you
can verify
um so i like the idea of verification
but i also think that
um just having having a ground truth or
not having a ground truth would be too
rough
of a of a categorization to not use it
um
because there are many applications for
instance in biology where you don't
really have a ground truth but it's
still very useful
to use evaluations based on our best
guesses
and then you know have now have a
machine like run
analysis for us just as an example
um so but i i will i will i will just
say that and i'll leave the question
because i think you've already said
enough about
context and that delegation and all that
okay thanks
great uh yeah one last minute cataline
are you still with us
i'm still with you yes yes decision
trees
i'll be quick so regarding your answer
to email's earlier question about
explainability
you said well in principle the decision
tree would be explainable if you
have a thousand layers yes the principle
of that is explainable but that holds
also for deep learning
so is that an essential difference
because of course a thousand layer
network
is not comprehensible for uh people in
general
uh wait you're saying that in the same
way that a decision tree of a thousand
layers is explainable so is deep
learning yeah
okay so i i that goes in
that goes against what i understand as
deep learning i mean
the whole problem with the whole field
of explainable ai is to attempt to solve
the problem that we can't explain
how deep learning algorithms come to the
outputs that they come to
so i but i'm not a machine learning
expert so i
uh i mean you can explain the
fundamentals of it how you construct
these things how do you work okay so no
so sorry the explanation that i'm
looking for is the cons
yeah you can explain there's many
different types of explanations you're
right
but the difference between the two in
one the decision tree
you can actually explain the features
and considerations which led to the
output which played a factor in the
output
and how and maybe even how much they
played a factor into the output whereas
in the deep learning situation
you cannot do that anymore that's that's
okay with that i would agree with
you you could ask back like why did you
make this decision and get a
comprehensible answer from a decision
player yeah exactly and so that's that's
what's really important to me here is
those considerations that played a
factor
yeah happy with that
yeah uh yeah the next uh question is
mine but unfortunately we don't have
time to
uh for that question and also for all
the remaining questions which i have to
admit are really
exciting so uh feel free to get in touch
with scott later
to discuss those i will so thanks again
scott that was a fascinating talk
thanks to everybody thank you again for
the invitation it was a lot of fun
thanks for the questions
i'm sorry we didn't have a chance to go
through all of them
okay uh see you all next week next week
our speaker will be jared
so looking forward to that


Robots Learning Through Interactions (Jens Kober)

there we go yes yay
thanks a lot for the kind of
introduction and also for the
invitation uh so i'm very much looking
forward to having
some very lively discussions uh as
hopeful
as most of the times um yeah
so nick already uh you know told you
about the question who should
help whom so most people if they think
about robots they would
think about robots supporting us with
the
tedious or dangerous tasks and then yet
very annoying cases where
the robot does something wrong or gets
stuck and
we also have to help the robot to get
unstuck
and then in a similar way who should
teach whom depending on
what kind of research you're doing or
what your perspective on robotics is
you might say yeah could be a great idea
to actually
use a robot also as a teacher for
humans my research focuses
more on teaching robots
new tasks so that's an example of
kinesthetic teaching where a person
shows
the robot how to solve a new task
and in this talk what i'm going to focus
on
is how do we actually make this teaching
in interaction with a per person as
smooth as possible and
as efficient as possible for the robot
so they're largely two different but
complementary ways on
how robots can learn new skills so so if
i say skills i'm talking about
learning some kind of movements on the
order of a few seconds or a few minutes
and the first one is imitation learning
where a teacher demonstrates a skill and
the student then tries
to mimic tech like we also saw in a
previous slide
here's an example from my own research a
bit
older and you'll see one my former phd
student demonstrates
the robot how to unscrew a light bulb
and then how to put it in a trash can
um so here yeah you need a couple of
turns to get it loose and then you move
it to the trash can
so just playing back the this recording
that's not really interesting what
you're really interested in is to
generalize the movement so for instance
in this case for the number of rotations
you need to do
to unscrew it but you could also just
think about changing the position of the
light bulb holder or the trash in
particular what we were interested
in here is the order of
little sub movements you need to do so
you need to move towards
the bulb then grasp it rotate it in one
direction
and so forth and then you can represent
that as
a form of a graph that tells you in
which order you need to do things
and then what you were also interested
in is okay when do we actually
switch to the next primitive
and then the robot can reproduce it and
also
generalized to new situations
so here you see it unscrewing and then
when it's it becomes loose
it's pulling slightly on the bob so you
can actually remove it
um so that's really important to detect
it accurately otherwise
the light bulb will go flying okay
so then it successfully did that
so that's an example of imitation
learning
and the other side is reinforcement
learning as you can imagine
like for us if the task gets slightly
more complex
also a robot needs to practice
what i'm going to use a bit as a running
example is ball in a cup so that's a
little game where you have to catch the
ball
in the cup and here's my cousin's
daughter she was
around 10 back then i showed her once
how to do that
and then it took around 35 times until
she
caught the ball in the cup so
reinforcement learning is learning by
trial and error and trying to
maximize the reward and you can see the
little uh rewarded chocolate
here okay so in both um
examples i showed you you have the human
in the loop
in some form or other uh for the
imitation it's very clear you have to
human demonstrating and
the robot mimicking but also for
reinforcement learning you need to
define
the revolt function some kind of
function approximation
and how exactly you set up the problem
and so forth so to illustrate a little
bit what we
did there is the reward is more or less
based on the distance between the ball
and the cup
the first time the robot drives it's
still a bit far
so gets a pretty low reward close to
zero
next time it's a bit closer so it gets a
bit higher reward
and if you imagine that you do some kind
of weighted combination of the two of
them
you might get something that's already
pretty close um
and then in this case pumping on the rim
of the cup and then
the maximum reward would be a one in
this case
so it's already pretty close and then if
you repeat that
so mainly focusing on the good one and
only the other two a little bit you
can hopefully catch the ball in the cup
very quickly so that's the robot trying
out different things
and then seeing what works what doesn't
work and adapting accordingly
also here we helped the robot
additionally by giving it an initial
demonstration
and so here there was a good
demonstration
but then as the dynamics are a lot more
complex here if you just play that back
then
you missed the cup by one of 10 15
centimeters
after 15 trials or so it's already
considerably closer
now the next one i think goes a bit too
far
and then it's going to hit the rim of
the cup
and then finally after 60 trials or so
it starts to get the ball into the cup
and after 100 tries or so that has
converged and works very reliably
okay so as you can see in the dates
that's
uh some fairly old videos um
what people are working on nowadays is
doing end-to-end learning so
we had some vision system that tells us
where the
ball is um we used low dimensional
representation and so forth um if you do
not use neural networks to try to learn
end-to-end
that usually means you need some form of
big data or some other tricks to do that
so this famous example here
that's from google they had an army of
robots working day and night for a
couple of
months actually non-stop just to learn
to pick up simple objects in bins like
that
so that's clearly impressive but not
really
desirable or practical in any sense
okay so what i showed you so far
is the human is involved in the
beginning
either giving demonstrations or setting
up the problem and then the robot is
off on its own trying to learn something
if you again compare it to how humans
learn you'll actually notice that
there's often a continued
student teacher interaction so while
somebody is learning you might provide
additional demonstrations or some other
form of intermittent feedback
and at least nowadays that's still
largely missing in robot learning
and i believe that including these
intermittent interactions with a teacher
allows you to
speed up the learning which in turn
allows the robot to solve more complex
tasks
and that's also a somewhat intuitive way
for
humans to teach and in the remainder of
the talk i'm going to show you a few
examples of
that's what i'm claiming here is
actually hopefully true
okay so how does this interactive
learning look like we have
a robot and an environment
the environment has a state so it could
be the position of the robot in the
world or the
position of the robot arm and the agent
has a policy which tells it
which action to take for each state
so depending on where it is what to do
and that's just more or less traditional
control loop
now what we have additionally is a
teacher
and the teacher is going to observe the
state of the world and the robot
potentially also what action the the
robot took
and then there are many different
variants you can think of but in some
form or other
the teacher is going to provide
additional information
to the agent usually if something is
going wrong so it could be an additional
demonstration to show something that the
robot hasn't seen yet
and could also be just a correction on
how to modify
it the robot behavior to perform the
task better
so in the first few things i'm going to
show you we're going to focus on
corrections in the action space so the
teacher very explicitly tells you
tells the robot okay you should have
moved a bit faster
or a bit slower um in order to perform
the task well
to come back to the example i had before
the ball in the cup so here you see the
same thing
however now we start learning from
scratch and carlos my postdoc
is sitting in front of the computer and
occasionally giving it some additional
feedback so in this case
move more to the left move more to the
right move slower
move faster and then after
17 trials or so it's already quite close
and then let's see
yep next one so that's after 25 trials
or so
it can successfully catch the ball in
the cup
so compared to the human gel there was
around 35 times if you ask an adult
that's also usually around 20 times
so it's really the same order of
magnitude now
and if you compare that to learning the
skill
from scratch so without initializing it
with imitation learning
just using reinforcement learning that
takes
two orders of magnitudes longer so some
2000 or so trials usually in our
experiments
and that's still not doing end-to-end
learning but relying on
low-dimensional inputs for the states
okay so that directly brings me to the
next part
learning from high dimensional inputs of
in this case from raw camera images
a typical thing you're going to see in
deep learning is
some form of encoder decoder structure
where you have the
high dimensional images here you bring
it
back to some kind of low dimensional
representation that
hopefully still contains all the
relevant information and in order to
train that what you
typically do is you decode it again
and then you try to ensure that the
output
matches the input so that effectively
here this
low dimensional representation contains
all the
important information and then
once you've trained this mapping from
the
camera images to a low dimensional
representation you can
use that to learn the robot behavior so
the policy on top of that
so then you would remove the decoder
part and then start learning
the policy either completely freezing
this part or maybe
also training that partially
so you could do that beforehand
so you collect lots and lots of data of
images the
the robot is going to encounter and then
try to find some kind of generic
representation
and then learn a policy on top of that
so that's what we see here
but as you can already imagine that's
probably not the smartest
thing to do because depending on the
task you do
some stuff in the environment might be
relevant and some other stuff
irrelevant and that might change
completely depending on which task you
want to get
so then the obvious thing is okay so can
also learn that
while so this representation while the
robot is learning the task
and that's exactly what we tried to do
there
to learn simultaneously and embedding
and the policy and then
so really the main objective you want to
do is to learn this policy so the
actions the robot takes but then as an
additional
criterion we have to this reconstruction
from the
input image to um in this case a
slightly blurred
image of a tree again
and again while while this whole
learning is happening
the robot is going to get feedback from
its human teacher not continuously so
not we are not remote controlling it
we're not teleoperating it
it's just the teacher is jumping in
occasionally to
fix some things
so what you see on the left is the
reconstruction so based on the
low dimensional representation and then
you can see that already very quickly
it learns something that represents the
the real images reasonably well and
again what you can't see
in this video is the human teacher
occasionally giving feedback via a
keyboard in this case on
what the robot is supposed to do in this
case
push this whiteboard swiper thingy
off the table and
let me skip forward a little bit
and that's that's some so that's in real
time here so we can
learn that really in a couple of minutes
come on
no
okay let's see here's another example
of teaching this little ducky town robot
to drive
and it's rodrigo one of my phd students
and you can see him holding the keyboard
and if you look very closely sometimes
he presses
buttons to teach it how to drive again
here the raw
camera images and the reconstructed
images and again after
just 10 minutes or so it learns how to
drive on the correct side of the road
and then here that's in real time
and here you can see that really learned
and that he's not using the keyboard any
longer to
control the little robot
yeah so that's not just taking the
images
compressing it down to something useful
and then doing
a control on top of that however for
many tasks you need some kind of
memory um
so you need to so either if you only can
observe
the images but you need to know the
velocity then you need to
have at least a couple of images you
might
act based on something you saw in the
past and for that
you can have a very similar structure
but here
additional a recurrent unit in a neural
network
so here a couple of toy examples i'll
skip over those pretty
quickly here the agent only observes
the images but not the velocity but in
order to solve the task
so for balancing this pendulum you
really need to know how fast it moves
and here we compare two different
methods
the one on top that's all method where
we use
corrective interactions so we give
corrections in the action domain
and on the bottom there was a different
approach where you need to give
additional complete demonstrations which
is um typically a lot slower
okay so that's again the the
reconstruction and the images and then
here
it's a little real setup and that's the
demonstrator is really learning to
do the control based on the images and
not
based on some internal states
wait come on and then here
corresponding robot experiments
we have the camera inputs the only thing
the robot sees is this shaded
white area there so it needs to touch
the the fake oranges so it needs to
remember okay the orange was here and
it's going to move there
in order to then touch them later on
and also something like this we can
learn in
in five minutes in this case two rights
and then here if you train it longer
after 45 minutes
it can really do it very accurately
i'm not sure if you can see that here um
here again you could see somebody doing
the teaching
here you see nicely where the
keyboard and here the task is to
touch the oranges but avoid the
the pairs
because we already trained a good
representation so going from the camera
images
to a compact representation
uh teaching this additional task is
something that can really be done
within 10 minutes and if you compare
that
to a more traditional deep learning
end-to-end approach
without having a teacher in there
we haven't tried just because based on
the toy examples we
saw that's again taking at least an
order of magnitude longer so
think about spending a day teaching
something like that
and the other big problem is if you
use these other methods you need to know
beforehand
what kind of data is required so it's
really up to
the human demonstrator to think before
and okay that's the situations that
could arise
i need to collect data for that i trade
the system you test it out i figure it
doesn't work
i need to collect more data and that
typically is a couple of iterations that
are required for that
okay let's see i have a couple of
questions
nick asks if do you want to just ask
directly
sure so uh or um i actually the question
like so
do you assume that the human corrections
are more or less optimal or at least
like
work towards more reward but and then
and also what like what happens if the
human
correction is actually not helpful at
all so how can we
how can you take that into account
okay so we assume that the human
corrections
are at least somewhat helpful it's it's
not a problem if
they are wrong or the human changes his
or her mind
but if the human is just demonstrating
nonsense
then depending on which setting you talk
about
in the imitation learning if it's really
purely
learning based on these demonstrations
then obviously there is no chance
whatsoever then it's just going to to
learn whatever you teach it
in the other example i showed where we
combine it with reinforcement learning
it's very unclear what's going to happen
it really depends on how you set it up
the robot has the chance to
to learn it correctly based on the
reward you have
defined and then how much the human
input
hinders the robot is is a different
question
if it's at least something more or less
sensible
so it might not be the correct task but
showing something like you're not
supposed to move very erratically and
shaky but smoothly without
showing the actual task that might be
helpful you can come up with the
other scenarios where it's really going
to to harm the performance
okay luciano
yeah so um yes so my
questions goes a little bit i was
thinking this is like one step further
if you teach like a human teaching a
robot
and but assuming that the robot is
stitching another robot
and let's think about like i don't know
some center of a warehouse but of course
if the
same robot with same joints okay they
could just transfer
the learning but let's assume that the
robots are different as
we from humans are different from the
robots the robot is teaching another
robot that's a bit
different so then i'm thinking of course
that could scale up and then
some emergent phenomena could come there
because if you didn't really
learn this is 100 percent didn't teach
someone else
yeah so how do you see and there's some
uh alternatives you do you think like
more imitation learning
enforcement learning what how could we
tackle this kind of issues
yeah good question i don't
i don't have a good answer
[Music]
so it really depends on what you want to
teach or what you want to transfer um
so if you i'm going to get to that in a
second
uh so if you okay i'll just show that on
the slides because that nicely collects
to what you were asking
so what i was talking about so far was
teaching
in in the action space so directly in
this case in the seven joints of the
robot and then
saying which velocity for instance you
you should apply
verge is kind of very direct and low
level and probably allows you to very
much optimize the the movement however
it's typically pretty high dimensional
and at least for humans it's
non-intuitive connecting to your
question
it doesn't transfer to a different
embodiment at all
um well if you consider the state space
or the
um the task space
so for instance the position of the end
effector then that's something that you
could transfer so if you
know the trajectory of the end effector
then it doesn't really matter how
the kinematics of the robot as long as
it has kind of the same workspace
obviously yeah
um depending a bit on the constraints it
has but that's something that's a lot
easier to
to transfer and arguably that's also
a lot more intuitive and easier for for
a person to teach
that isn't familiar with the robot the
downside then is that you still need to
translate that into
the the movement of the robot so the the
actions
again i'm using some kind of inverse
kinematics so
dynamics models which might be as
themselves a bit tricky to learn
so to come back to the question
so if you have robots with different
embodiments
thinking about at which level of
abstraction you should to transfer is
definitely one
thing that that's done and so for
instance
transferring trajectories in end effect
then in joint space would help
and the
i mean the big advantage
robot to robot teaching would have is
you can work around the the
communication issue
a lot better so you have complete
knowledge
on both sides effectively
so i've i would say better ways of
doing that compared to what i was
presenting
here which really also focuses on okay
a human doesn't really know exactly what
he or she
should do it's they don't like to kind
of
be constantly tell they operate the
thing but
it's a lot more um pleasant if you only
need to jump in when something goes
wrong and occasionally give feedback
and you really care about how
how long it takes so um
for robot robot teaching you can
probably do a few
other things that
might be better so yeah you could
probably apply those methods i'm not
sure if they're the best
yeah okay thanks
okay and then how could we what did he
have how could we keep
human control yeah i was thinking like
if you teach but you don't teach like
hundred percent
uh the robots don't download 100 what
you mean
and then they teach someone a dozen 100
so you gotta
increase the gap as far as you go from
the original source
how is that called that stupid game
where you whisper in somebody's ears and
then you do the chain
yeah it's like a telephone game i think
right chinese chinese whispers
chinese whispers yeah exactly so you
would get something like that
indeed yeah
so yes i'm to be totally frank at the
moment very
much looking into
the question how can robots
be best taught by humans um
and it's a bit more on the algorithmic
side so we're looking into how
humans experience the teaching
uh but it's not like we're at the moment
looking into human robot interface
design so to say
um and then
what you're saying that adds no whole no
new additional layer of complexity on
top of that definitely very interesting
and
um we should get there
um okay what else
so wendelin asks do you want to unmute
and directly ask
sure so i was just like referring to the
discussion before um so if if
the robots or the first question is like
the robots still get the original
environmental reward right
yes in the reinforcement learning case
so
um would this just mean that if you have
a robot that trains another robot
after a while that other robot would
still
uh converge to the original task it just
made it take longer because the first
robot might have given him some bad
ideas
could be yeah
yes i mean that's the general question
in
all transfer learning uh approaches
when is it actually beneficial to to
transfer and
when is it better to to learn from
scratch
i mean like it's it's very different
from transfer learning
because in transfer learning you like
for example is very different from
invitation learning in imitation
learning
you don't know what the real task is but
in this case i think
like luciano's concerns
are should not be too hard
to disprove because like basically um
you still get the original task
so you can't really stray too far away
from that
yeah i agree i mean so i've been
presenting two different
things one some of them were purely
based on imitation learning so
supervised learning where you
don't have that and there kind of this
drift
could very well occur but but yeah i
agree
if you think about reinforcement
learning
and transferring the
the reward function as well then
yeah worst case is it takes longer to
to converge to that okay thanks
okay good
so
here um i was just introducing that
that we might actually want to teach the
robot
in in state space so in the end effector
space in this case
and in this drawing i have here that's
pretty obvious because that's something
that's
already provided more or less by the by
all robot manufacturers
but in some other cases that might
not be so obvious so um i have another
little example
here is there's my mouse here
is that the laser pointer uh on on the
robot
and it's trying to to write a letter on
a whiteboard just a
little fun task where we actually
don't know the model of the
the complete system and we actually need
to learn it at the same time
as learning the task um
or you could say before we do the
learning somebody
sits down for an hour half a day and
writes that down
in order to program the robot to
be able to be controlled in in
task space so in this case what i mean
by task space is
what you would like to do is to directly
control the
position of the laser pointer point on
on the
whiteboard and not care about how the
robot actually is supposed to
move and the approach
snehal came up with actually allows to
learn that simultaneously so learning
the
mapping from the task space to the robot
actions and actually how the the task is
supposed to do
again using yeah
using interactions and here that's
one of the initial
things where you still have lots of
corrections from the human teacher
in the top right it's just the zoomed
one and we
did some enhance the the laser pointer
and then here it
after 15 minutes it again learned to
do that and the last move you saw
here is the robot doing it
autonomously
so here's writing core the name of our
department
which nicely coincides with the first
three letters of the uh
the conference where he published it
okay um so so far what i was talking
about
was very much on the
level of directly controlling the
trajectories if you like
of the robot um
in the very first example i showed you
with the light bulb
unscrewing it was on a higher level
everywhere i'm
considering the the sequence
so for the for learning these kind of
things in a sequence
what you will quite often have is
different reference frames so that if
you say
you have an object and you attach a
reference frame to that
the robot can move relative to the
object rather than
moving in in world coordinates or in
absolute coordinates
which then very much helps to generalize
to
no matter where you put the object and
if you start having more
objects and then suddenly you get a
whole
lot of these reference frames or
coordinate
example would have to light bulb the
holder the um
the trash bin maybe volt coordinates
additionally
and then one of the challenges for the
robot is to figure out from the
demonstrations
which is actually the relevant
coordinate frame so should i move
relative to the light bulb or the
holder or should i actually use position
control or should i use force control
so here's another example of that
where the robot is picking up this mug
and we have two
coordinate systems one is relative to
this coaster and the other one is
relative to
the cup and now if we only give one
demonstration when the cup is on the
coaster
there's just not enough information in
the data we collect
for the robot to figure out what it's
supposed to do
you could put in some prior okay we're
going to touch the cup so that's
probably the one we're interested in
and then use that but
you can do that but that breaks down
pretty quickly
because these priors tend to be somewhat
specific
so now if you always have to cup
on top of the coast that's not a problem
it will work fine for
for either reference system because it's
always
just a little offset and no matter how
you describe the movement it's going to
result actually in the same position
now if you separate them like we have in
this example here
then suddenly that becomes interesting
and the robot needs to
decide on what to do if it's
purely based on the initial
demonstrations then there's no way it
can do it
other than flipping a coin if it can
ask for feedback or have interactions
with the teacher
then that some something that is easy to
resolve
and what we were additionally
considering here is like i described
before in some cases you actually don't
care about this ambiguity so it's fine
if you don't know which
of the two you want to use because it's
going to result in the same movement
and you don't want to bother the human
teacher if it doesn't really matter
anyhow
so the approach we came up detects
if the well obviously if there's still
multiple reference frames possible
if it's actually relevant in the
situation we're currently in
and then if it can't decide on its own
or
if it's actually relevant only then the
robot is going to
request feedback
okay so that's the corresponding video
here
um the demonstration was already given
so let me go back that was way too quick
so we demonstrated picking up the
cucumber moving it
to the lower tray here
so if it's in the same position
then that works fine if we move the
cucumber only that's also fine because
it's
we touched that and we know that it's
going to be relative to the cucumber
okay so then we can remove that
works fine now what happened is we
switched to two crates
now you have an ambiguity so are we
very interested in moving relative to
the world coordinates so the absolute
position somewhere here
or are we actually interested in in the
crate and what you're going to see now
is it starts moving and then
it stops because it's unsure which is
kind of
the trigger for the human teacher to
give feedback in this case it's a
haptic kind of little push in the right
direction
which then helps the robot to
disambiguate that
and from now on it actually knows okay
we're not interested in the
in the absolute position but in the
great position
uh so it discards that from its
decision tree so to say and from now on
has
learned how to act in these situations
okay you can make a little bit more
complex if you have multiple reference
frames
attached to the two sides of the object
of this box i mean and then again the
question is should we move relative to
that one that one or are we actually
interested in
the one that is bang in the middle
between those
yellow and red coordinate systems
so here you see a different form of
interaction wires and kind of
on screen yes no interface so i was
asking
the yellow one and giovanni said now
the apple one also know and then the
only remaining possibility was
something in the middle
so here's a one last task where we're
supposed to
stack boxes on top of each other and
based on that we also
did a little user study where the 12
people i think
uh just before the lockdown happens
um so here the task is just like
one specific couple of a box of tea bags
on top of
another one here you see
so on the kinesthetic demonstrations
and what we compared here is purely
doing it with these
demonstrations versus having
the the interactive corrections or the
robot actually asking for for feedback
if there's an ambiguous situation
and as you can already see if you've not
done that before
teaching it like that is
also not so easy time consuming and
annoying as well
so there was you need like six
demonstrations to
get all the combinations covered well if
we
do that with the interactive method here
you just get one initial demonstration
and then only in some cases
it's not going to know what to do and
i'm going to ask for
again feedback by by getting pushed
so that's significantly quicker to to
teach
for the human and to learn for for the
robots
and also in terms of mental demand
physical demand
temporal demands if we ask the
participants
they very much prefer to teach
in this interactive corrective fashion
rather than just giving a whole lot of
demonstrations
and yeah you can see that for all the
scores the
this lira which is the interactive
method is doing a lot better than
the kinesthetic teach in which is just
giving a whole lot of demonstrations
beforehand
and hoping that you covered all the
situations you
need to do okay
so that's the end of my presentation no
robotics presentation without a little
video clip from a movie this is from
real steel where you can really see
how teaching a robot might be child's
play
if you have the right interface and
that's
what i'm trying to work towards okay
so to sum up i'll just sum up quickly
and then i hope there are a few more
questions
um so i hope i could show you a few
examples on how
doing this teaching interactively and
intermittently
can help speed up robot learning i
showed you
a few different variants of using
imitation learning or combining
those demonstrations with the
reinforcement learning
and then like i was saying before
there's still a lot of open questions
how do humans like to teach
um and then especially for this audience
what i'm presenting is that actually
meaningful human control
yes you can teach the robot to
act like you want so it's it's a form of
human control but then still
there might be quite a few things
you're unsure about and how it's going
to generalize how
how it's going to react in in different
situations where you have not
taught the robot good
thank you very much jens for this really
interesting talk so yeah
the well the the questions are already
rolling in
if you want to ask your question please
go ahead
okay um hi uh thank you
okay hi okay so
that's where the talk was very
interesting um
and uh yeah my question goes uh do you
formally model
any of these interactions or uh learning
processes
um so
um my i just started my phd
on uh mental models uh
context and um
yeah so i'm looking for ways of modeling
this uh
team interaction to achieve a certain
goal with context awareness and all that
and this is very interesting because
this can also be seen as teamwork
uh if you are to achieve a goal
that both agree um so i was just
wondering if you
look at the uh at the knowledge-based
models
of any kind or if you're just looking at
more machine learning uh models and
results
um i'm not sure if you understand
what i mean not entirely sure
we'll see that's a discussion um
so we
modeled these interactions
in indeed more kind of as a machine
learning
approach in the reinforcement learning
things i showed for those people that
know reinforcement learning
the the human kind of interaction was
actually modeled as a form of
exploration to
be able to incorporate it in the
reinforcement learning
framework in the for the other things
it's effectively some kind of a switch
in the robot logic that tells it okay so
here was
an interaction uh so i need to treat
that
differently but we're not
so the robot itself knows about
its behavior or its movements where
it's sure or unsure about what it's
doing
so so in that sense it's modeled and
taken into account
but really the the interaction
is more kind of on on a human
level that we don't model that very
explicitly at the moment
but if you have ideas on on how to do
that and how that would be helpful
yeah i'm i'm wondering yeah uh for
example uh
because yeah so i just
saw for now um yeah so this moment when
the robot can detect that it's unsure
about the box for example it stops for
an interaction
and yeah i was wondering if you have
some kind of
if this is formally modeled in any way
so
that there's actions and states and it
reached that state and then
it stops and waits for the interaction
and then it goes back to
so how that's modeled this was more my
question maybe
so how is i'm not sure if you're talking
about the same kind of modelling
so what what it does internally is it
detects
that in this case okay i still have two
possibilities left
and the actions i would do according to
each of them would be contradictory so
in one case i would move to the left and
in the one case i
would need to move to the right so it
knows
that there's a problem and then there's
a
switch in the program that says okay now
i'm going to stop and reach
and request human feedback and once i
get the human feedback
and then hopefully it will allow me to
choose to correct one
and then the robot can continue
okay thank you i will also think a bit
more about it
yes it's on my website ikari
i can send it to you directly thank you
great ah this channel yeah
no one has i have a it's kind of a
follow-up on a previous question but
it's on the
on the point so i just first wanted to
understand if i got right what is it
what is the trigger for the robot on the
camera like to define
which we should ask for feedback
and this is yeah that's the so true part
that's the first part and the second
part would be
thinking about the context that the
robot would be teaching another robot
would be a similar point and say okay
now i want to intervene because i see
you're doing
something weird or
yeah so first is
what was the point how do you define
again the the point that the robot asks
for feedback
okay so
let's take this example again because
it's a bit easier i think
so what we already have is one
demonstration
and then we can represent
the movement in
two different ways one is relative to
this
coaster and one is relative to the cup
so we know we have two representations
of the movement and we don't know which
is
the correct one yet because we only have
a single demonstration
if we encounter the same situation again
maybe we just moved to two objects
together to a different location
and then we the robot checks okay how
would the movement look like if i move
relative to the cup and how would the
movement look like if i move relative to
the coaster and if those are very
similar
then you don't really care uh you don't
need
to ask for feedback because you can just
do it right
okay yeah now the other situation is
i separate the two objects and then
again if i predict how the movement is
going to look like if i move relative to
the coaster or relative to the cup
then you're going to discover okay those
are very different movements and they're
not compatible
yeah which and then you then the robot
is going to ask for
for feedback yeah yeah that's
yeah that's clear for me but my wonder
is like more about the intentions there
like when you had
this box with the cucumbers and the
tomatoes you don't know if you want like
the
a specific absolute position or the
other one so that's like
even though the coordinates everything's
still kind of the same but your your
is the intention for the human that you
don't really know
so so the intention in this case is also
modeled as
the uh as these reference frames if you
like so the intention
would be either move relative to the
cards uh the cucumbers move relative to
the world coordinates
okay so still as a reference frame yeah
yeah i understand that
so there could be if there's a deviation
for the reference frame that's a good
point of asking or giving
feedback as soon as there is some
deviation there yeah
so in this case we focused on on
reference frames because that's
something that occurs frequently and is
pretty visual
but i mean you could do similar things
also for
for other types of ambiguities so in
particular what i'm interested in
is uh force and position control which
is also
if you always have the same object
doesn't really matter what you do
because one
kind of results in the other and the
other way around
um so it's really about
predicting different possibilities and
figuring out if the
um contradictory which is
what we call an ambiguity
yeah that's great okay and do you all
see that
yeah okay thank you
thanks so you have another question
no i think that's it let me leave also
the opportunity for someone else to jump
in
so so just for my quick clarification
for myself
so basically so so you could say more or
less that you know one one
big issue in meaningful human control is
that what if we misspecified the
objectives of a robot or ai
then for instance this your this this
resolving ambiguities would be
one way like that the robot would
inherently stop and ask for feedback at
the moment when it's not sure about what
what
the objective should be is that a
correct would that be a correct
extrapolation of this work
i'm not sure in the sense that
if you misspecify something i'm not i
don't think
the method i presented will solve that
because i mean if you explicitly
specified then the robot is going to be
sure about it what you could do
additionally is that you allow
also so in the last part i presented was
effectively
robot driven questions or interactions
you and the first part was more about
human driven
interactions if you assume that you have
your misspecifications
and you allow human-driven interactions
so that the human can always jump into
to change or correct or something then
then yes that might be
a way to at least detect that something
is going wrong and
potentially also to fix it
thank you so uh we i think we have
question for
one more time for one more question uh
we're almost approaching two o'clock
yeah
if they're more questions shoot me an
email
great so
if there are no more curses indeed uh
yeah
i would like to thank you jens for this
this great and interesting talk it was
very uh very inspiring
uh and yes and uh also thank to all the
the audience for being here and
interacting with with us and the ends
and uh
yeah we'll see you next time
bye-bye thank you very much
yes thank you
stop recording