Sven Nyholm: Responsibility Gaps, Value Alignment, and Meaningful Human Control over AI

good
so the recording is on so today's
speaker is
sven newhom he's an assistant professor
of
philosophical ethics at the utah
university
his main research focuses on ethics of
technology ethical theory and history of
ethics
so he has written on many things i think
the first time i
saw him uh give a talk it was on kant's
uh universal law i was in
ironia that was seven years ago but he's
now mainly writing on many different
things self-driving cars humanoids
robots a dominus weapon system
uh if you want to read more about his
ideas
i can really recommend this roman and
littlefield's book
humans and robots uh
so it's a really great book um today's
talk is titled responsibility cap's
value alignment and meaningful human
control over ai
and i would like to give the floor to
you as twin
well thanks a lot harman now and
uh yeah i mean i am going to talk a
little bit about meaningful human
control but mostly about the first
two topics because i uh i mean i have a
papers of people that
are interested in reading a written
version of this uh
please feel free to get in touch with me
and i would very much appreciate any
feedback people might have so
just send me an email can you now see my
slides yes
great thanks okay so responsibility gaps
value alignment and meaningful human
control over ai
uh we're gonna start with lisa doll
uh who as you will probably know i lost
uh
against the computer program alphago uh
they played five games and he actually
won
one of them but the other four games
were won by the
computer program that had been training
against itself
and so although there was a human who
was sort of moving around the pieces on
the board
the guy on the left there i mean he
didn't understand the strategies
of alphago and so he was a little bit
like the person in
john searle's uh chinese room experiment
i mean so
if you know about that experiment a
thought experiment
uh john searles imagine someone who gets
instructions about how to put together
messages in chinese and without knowing
chinese the person can do that but they
wouldn't know the meaning
of those messages and this person who is
kind of playing go against
the world champion it's a little bit
like that i mean he's
moving the piece around on the
suggestions from the computer program
doesn't know why exactly doesn't
understand the strategy
but then the computer program won the
game listed all incidentally retired
after this
he didn't want to play go and morph they
felt meaningless
another case that you may have heard of
uh quite different exactly two years
later
so lizadol was beaten in uh by alphago
in 2016 in march
in march of 2018 uh the first time
uh it happened that a person was hit and
killed by a self-driving car
so there's a pretty gruesome video of
this and so the safety driver
did not react in time uh nor did the ai
system in the car
uh it classified the person first i
believe it's a road sign then as a bike
because she was walking with a bike then
as a person then i think it's switched
back and forth between classifying her
as a bike and a person that just an
appropriate action was not taken
the safety driver did not react in time
and the woman
elaine herzberg was hit and killed and
died on the way to the hospital
uh some people worry about not having
control not understanding
ai systems uh some people even say that
if we start using ai systems
to create what they call killer robots
autonomous ai driven
weapons systems this might be another
case where we lose control
we don't understand what's exactly is
happening in these systems
and so we cannot be held
appropriately responsible but people
should be held responsible for
things that are both good and bad
outcomes
so we might have uh problems with what i
will call
well it's not just me it's a commonly
used term responsibility gaps
but what i will do uh is to
try to identify sort of broad four broad
classes of responsibility gaps uh
this is the the fourth one i think i
haven't seen people talk about before
and i would relate those types of
responsibility gaps to the issue of
value alignment and then if there's time
i will
talk a little bit about meaningful human
control but it's all going to be about
meaningful human control
indirectly and we can discuss that more
during the q a
so don't worry if you wanted to talk
about meaningful human control
that's going to be lurking in the
background throughout the talk
okay so i should first say what is meant
by responsibility gaps more generally
uh and it doesn't have to be anything to
do with ai it can be
uh with large groups of people are doing
things together
uh they can be good or bad effects but
uh it could
can happen that maybe it's the culture
of the
organization uh maybe it is some other
group level effect that means that it's
very hard to find some individual person
or maybe even group of persons who can
be held responsible but there is some
outcome
that you think it would be appropriate
to hold somebody responsible for
and so there's a responsibility gap and
so related to ai so if there's an
ai system bringing about an effect and
it seems that it's a good and bad effect
you want to hold someone responsible
but you can't find anyone who it's
appropriate to hold responsible
then we have a responsibility gap
okay so in order to get to my four
kinds i'm gonna use two general
distinctions
from a more generally fl a general
philosophy of responsibility
the first distinction is between what i
will call
negative and positive responsibility
so negative responsibility i mean
something bad has happened or someone
has
done something bad and we want to blame
them or punish them
negative uh or something good
has happened and we want to praise them
or reward them
that's another way of holding someone
responsible giving them
credit to credit where credit is due and
that is what i mean by
positive responsibility another
distinction is between
backward-looking and forward-looking
responsibility
so uh after something has happened uh so
someone has
been injured or some good outcome has
someone has been saved let's say
we want might want to blame or or praise
someone for what has
has happened but we might also think
that
certain people have responsibilities to
take precautions to avoid
certain good outcomes or perhaps to
promote certain good outcomes
so looking at at the future uh who is
responsible for making sure that things
go certain ways
rather than others that's what's meant
by forward-looking responsibility
so if you have these two distinctions
and you're
thinking about responsibility gaps you
can create a sort of classification
matrix that would look something like
this so
you could have responsibility gaps that
are
backward looking and negative uh
forward-looking and negative
or backward-looking and positive or
forward-looking
and positive so those are the four
cancer responsibility gaps i want to
talk about
and let me start by just kind of mapping
this on to some of the existing
literature
i mean as i said i have a paper version
of this and i go into some more detail
about how to kind of
map the territory here but here are just
some examples
i mentioned killer robots autonomous
weapon systems earlier
and there's a famous article by robert
sparrow that maybe a lot of you have
read
when he argues that there might be
responsibility gaps related
to them and that discussion is about
backward looking negative responsibility
gaps after something bad has happened
it's unclear who can be uh blamed
now and this is pretty much all the
discussion about responsibility gaps
is almost all about backward looking
negative responsibility gaps so john
diana her for example has a paper about
what he calls retribution gaps who
should be punished if there's something
bad happens
and we can't find a appropriate person
uh in a recent forthcoming paper by
uh one of your one of your colleagues
philippo santorini the
ceo together with uh julio mccachey
they talk about what they call
culpability gaps
moral accountability gaps public
official counter accountability gaps
those are all backward looking negative
responsibility gaps
what about positives forward-looking
response sorry
what about backward-looking positive
responsibility gaps well
uh john dana her and i have written a
paper
about uh the automation of work and it
could be that if more and more
work tasks are
made outsourced
to ai systems and robots
then maybe the room for human
achievement in the workplace is becoming
becoming smaller and smaller and so
there might be good outcomes
uh so there might be an ai system that
identifies something as a cancer or
something like that because that's
well that's not a nice outcome but it's
nice that there's a diagnosis
maybe the ai system can even recommend
the treatment uh
so that something good may have happened
uh because the disease has been found in
a treatment has been recommended
but perhaps no human can sort of claim
credit because it was all done by
machine learning and uh in a way that's
opaque to
people involved there might be kind of a
gap in achievement
uh what about forward-looking
negative responsibility gaps well in the
book that
hermann very kindly recommended earlier
humans and robots
i talk about what i call obligation gaps
and so
let's say that self-driving cars are
driving and it's it should
not hit anyone uh but it cannot be held
responsible if it were to hit someone
and so after the fact there might be a
responsibility gap
uh of a backward looking kind but if
that's the case there would seemingly
also be a forward-looking responsibility
gap because
it it's not gonna be true that if it
hits someone
it can be blamed and perhaps the person
riding in the car can also not be blamed
so there's a worry that there's a gap in
terms of who exactly
is obligated to make sure that no one is
hit
i already mentioned felipo and julio and
they have another
gap that they call active responsibility
gaps and i think
i'm not sure but i think that that
corresponds to what i'm calling
uh forward-looking negative
responsibility gaps now
the box that i am sort of most
fascinated by and
i want to talk most about in this talk
is the last one
and uh as far as i know in the
literature people have not
discussed this type of responsibility
gap uh so who should make sure that
something good happens in the future
who is responsible for that
uh now before before i get to that
let me talk about two asymmetries that i
think there might be in terms of how
willing people are to fill these gaps by
sort of taking responsibility
by stepping forward and saying okay i
will be responsible
first of all i think people are quite
willing to take responsibility for good
things that have happened in the past
but they're quite unwilling to take
responsibility for bad things that have
happened in the past
that's one asymmetry that we have to
deal with and people might
want to take responsibility for good
things that have happened even though
they're not
sort of justified because they didn't
actually do anything very impressive
uh whereas if people have done something
bad and they deserve to be punished
or blamed they are usually pretty
unwilling to be
held responsible so that's one issue
here another issue is when it comes to
uh forward-looking versus
backward-looking
positive responsibility uh people as i
said are quite willing and eager to take
responsibility for good things that
that have happened in the past however
when it comes to making sure that things
go well in the future
people tend to be uh much less willing
to take on responsibilities
now all of these asymmetries or both of
them i think have to do with the costs
that are involved in taking
responsibility for something
uh if you take responsibility for
something good that has already happened
there's just uh things to be gained from
this you get praise you might get
rewards etc but all other responsibility
gaps
involves taking on certain costs even
taking on the responsibility of making
sure that good things happen in the
future
can for example involve an opportunity
cost uh because you have to you know
work to make sure that the good outcome
is achieved and thereby
possibly uh missing out on other
opportunities
to perhaps promote for example your
personal interests
okay so it's pretty hard to fill these
responsibility gaps by just expecting
people to kind of step forward and take
responsibility
uh now what other reasons are the wire
responsibility gaps arise
uh it's quite common to say that uh ai
and robots can become a sort of agent of
some sort
and uh if they're acting in an
autonomous independent way
these systems or these agents then uh
and
humans don't are not able to control or
predict what they're going to do then
that might be one reason why you think
there are responsibility gaps
uh i mentioned robert sparrow he gives
the kind of argument in terms of
when it comes to military robots another
reason
is the problem of many hands so this is
where
to make one ai system or some other
thing
at work it sometimes can be required to
have lots of different people involved
performing small tasks and parts and
everyone might have a little bit of
responsibility but it might be unclear
who
bears the most responsibility and so the
responsibility might be sort of watered
down
and spread out too much over too many
people and therefore there might be a
gap
and lastly this is what i want to talk
most about
is that not only do people shy away from
taking responsibility for making sure
that future good outcomes
are uh achieved and this is the to do
with this empty box that
i uh you know want to fill with the new
kind of responsibility gaps that was
down in the corner
uh it's also the case that common sense
morality
doesn't uh postulate as strong
positive responsibilities to create a
good future as it creates response
as it posts responsibilities to avoid
harming people
and so uh this is discussed for example
by julian
savilescu at the bottom there and ingmar
person at the top
uh where in their book that's called
unfit for the future where they worry
about how common sense
uh morality might undermine our
sort of abilities to meet the challenges
of the modern world
they argue that we have developed uh
throughout you know
human evolution certain attitudes and
one of the attitudes that we have is
that we feel
and i'm calling now intuitively
responsible uh
much more for the harm that we cause
than for any benefits that we failed the
cause
and that we feel that also that we have
moral duties and obligations to not
harm but not necessarily in the same way
to benefit
and so uh people
are in common sense morality sort of
permitted to promote their own interests
and not necessarily responsible for or
required
to promote the overall good now what
about in ethical theories
is that the same and what about for
example in utilitarianism that says that
we should
should promote the overall good um well
actually even if you look at
utilitarians they tend to shy away from
saying that this
sort of positive responsibility that we
have uh
both bentham and mill for example in
their utilitarian theory say that
the overall good is best promoted if
everyone gets to promote their own
interests so long as they don't harm
other people
and a lot of contemporary
consequentialists say that a lot of the
values that we have
are to do with ourselves and our loved
ones and so maybe the overall good is
best promoted if everyone can kind of
promote their own personal uh
interest or the interest of their
children and friends and family
and so even utilitarian theories don't
sometimes postulate a strong
responsibility to promote
the overall future good what about
county and ethics well canton ethics
also has
a duty to promote the good but there
too can't describes this as a what it
calls a wide imperfect duty
you should have this as one of your aims
but you're not positively
responsible for spending all your time
on promoting other people's
good now i think there's an argument
that we can make for why
uh this creates a problem for a i value
alignment i think
this audience will all know what i'm
talking about when i say value alignment
but let me just remind you this is the
outcome where
we create ai systems that promote uh
human values that fit with human
uh preferences and that further our
interests
and when people talk about value
alignment they often talk about
trying to create more and more advanced
ai systems that hopefully will
promote human values and the good now
so if it's true that common sense
morality and many moral theories
they avoid and they involve strong uh
responsibilities
to avoid harm but not necessarily strong
responsibilities to activist drive or
positive outcomes
and ai alignment is the creating
creation of good future outcomes
there might be a kind of responsibility
gap of the forward-looking positive kind
here
uh because again if common sense
morality and
many moral theories don't say that we
have as strong responsibilities to do
good as we do to avoid harm
and ai alignment is a way of doing good
then there might be a kind of
responsibility gap in terms of who
exactly
it is that should be promoting the good
of ai
alignment okay so uh
one way then in which we can maybe fill
in this missing box
is to say that uh the ai alignment
issue especially future directed ai
alignment of you know
systems that we don't yet have but we
hope that someone will invent
that might be one case where we have an
instance of what i'm calling a
forward-looking positive responsibility
gap uh it would be good
if we achieve this goal but it's unclear
who exactly
it is that has a positive responsibility
to strive towards
fulfilling this goal uh
okay so uh now what about the literature
about
how to fulfill responsibility gaps could
we somehow use
that i myself uh in one paper from 2018
and also in the book
uh that herman mentioned i have argued
that we can view
humans and ai systems and other
technologies as forming a kind of teams
uh
and uh the maybe the the technology part
of the team is not responsible but the
human parts of those teams are
responsible
and you know who are these human
technology teams well we can ask
questions like you know who has the
ability to stop the technology
and turn it on and off uh who has an
understanding of how it works
who is able to update or request updates
to the technology
uh who is able to monitor the technology
and so on and so forth so if we have
these kinds of questions
and we have a lot of the answers
pointing in the direction of a certain
individual or team of individuals
then we can say that those are the ones
that are collaborating with the
technologies and that could be
held uh i already mentioned philippo and
julio they have uh what they call track
and trace theory i sometimes like to
call their theory the
the the package delivery or personal
delivery theory because it sounds like
you order something
you know online and you're waiting for
your package to arrive the tracking
condition is that the technology should
uh track human values uh and
i mean it sounds like value alignment it
should align the human interests and
goals
and the tracing oh i see that there's a
typo here should be tracking trace and
not track and track but
uh that's what i meant not a tracking
track but track and trace
the tracing requirement is that there
should be people who understand
uh the how things work and also the
moral significance of using the
technology in question
so i mean there's really an overlap
between what i had suggested and what
they are suggesting because
i i had also asked you know who was able
to uh
to understand at least on a macro level
how the technologies in question work
uh and whose interests are being uh you
know promoted by the technology
that's very similar to the track and
trace uh
conditions that they have so i i think
of our
respective theories as both being uh you
know
variations on the same theme basically
so those
are attempts to fill responsibility gaps
uh
but but they wouldn't necessarily
fulfill the forward-looking
responsibility gap that i've been
talking about since
uh that is to do with value alignment
and what
for example felipe and julia call
tracking
that just seems to be value alignment
and so uh there's a question of who
should bring
about that uh tracking or value
alignment
now what about uh rethinking uh
these responsibility gaps well perhaps
you could for example say that okay
there are not uh strong to promote good
outcomes
but uh if you don't promote good
outcomes then bad
outcomes might be brought about and so
you can kind of maybe
twist turn things around and say the
failure to promote the good outcomes
would bring about
a lot of risks and we do have a strong
moral obligation to avoid creating harm
and so
by maybe promoting the good outcome the
value alignment could be a means to the
end of avoiding
bad outcomes or certain risks
uh now i think these are all interesting
and uh i mean in the paper that i when i
talk about this
in some more detail i say that these are
really they're not so much practical
solutions as they're sort of
theoretical idealizations uh things that
we
can in theory do but in practice it's
actually quite hard
uh if you apply these sort of uh ideas
to cases like the
uh uber car uh hitting and killing uh
that uh
pedestrian elaine herzberg uh you know
uh alphago winning the game
you know how exactly this uh figures
apply to those cases and who's
responsible who can get credit for
winning a game or
uh be blamed for uh you know a
self-driving car uh
hitting and killing a person there might
still be a many hand
problem involved there might be uh
worries to do with
again who should make sure that there's
value alignment in the first place
and even if you are able to sort of in
theory reconceptualize
uh certain uh promotions of good
outcomes as
avoidance of bad outcomes and risk
management
that still doesn't necessarily point to
a particular person or set of persons
so i think a lot of the suggestions in
the literature about how to fill
responsibility gaps uh are interesting
but they're really
uh a kind of theoretical uh
idealizations that point out
relevant features of responsibility but
that don't necessarily give us a kind of
checklist
for how in every case we can easily fill
responsibility gaps
and again i especially worry about what
i call forward-looking possible
responsibility gaps related to ai
value alignment okay so i have talked
about some different cases
there are you know bad outcomes such as
people being hit by self-driving cars
that are what you might think of as a
good outcome you know an ai
system within a game of course in that
case it's not clear that it was actually
a good outcome because
as i said the human world champion got
he became so
uh disillusioned that he quit playing go
and
and so i i'm not even sure that that was
a particularly good outcome
and that there was any what you might
call value alignment in that particular
case
uh and so i want to say that we
shouldn't only talk about
backward-looking negative responsibility
gaps which has been the
normal thing to discuss in this
literature we should also look at
forward-looking negative responsibility
gaps and
backward and forward-looking positive
responsibility gaps
who exactly if anyone deserves credit if
something good is
done by an ai system but more
importantly who
has a more responsibility to make sure
that we create
good air systems that align with human
values
and uh not bad ones that clash with them
and
uh since common sense and moral theory
uh postulates much
stronger responsibilities to avoid bad
outcomes than to promote good ones that
we might have
a interesting and uh confusing
forward-looking positive responsibility
gap here
all right so thanks a lot again if
people want to read a written
version of this please feel free to get
in touch with me
i would very much like to hear uh
feedback on that too
so but but for now i'm very much looking
forward to discussing this
in this version of the material with you
thanks
so thanks friend uh they were really
interesting
so really nice how you connected the
value alignment with the
problem with responsibility gaps i think
that's a really really nice
novel contribution as well uh so
um the floor is now open for questions
so i see one question in the chat by uh
full court uh so if you're still
here please feel free to ask your
question
um
maybe he's already sorry
so i think he left so i can read out the
uh the question as sven
and maybe you can can answer and then
uh we can all benefit so the question is
when you first mentioned negative and
positive responsibility i
expected it to mean responsibility that
something bad does not occur
or that something good does occur how do
you think that this distinction might
help in thinking about responsibility
gap it might be
more useful than thinking about the
result of taking responsibility
what seems to be the meaning you attach
to post
to negative positive
yeah so that sounds as if what i was
calling forward looking responsibility
is what they were referring to and it is
true that
uh some philosophers like vernon
williams for example has used the
expression negative responsibility gap
to mean avoiding the creation of certain
bad outcomes but
uh i mean these are just
since a lot of people more recently have
talked about forward-looking
responsibilities to refer to that thing
uh and they have mostly then talked
about uh forward-looking
responsibilities
gaps of the negative kind i i see this
is just a
issue of terminology but i was certainly
talking about that thing
that the person in question wanted me to
talk about when i talked about
negative forward-looking responsibility
gaps
good yeah so that's that's how i
understood this well thanks
so i think the next question is from uh
uh maximilian kino
um oh yeah thanks very much thanks when
i i really enjoyed the talk i think it
was great um so i
i'd like to ask you about the definition
and the significance of the
responsibility gap to understand your
account a bit better
um so i think it's kind of assumed in
the literature that the mere absence of
responsibility
is not enough to constitute a
responsibility gap so
we are not responsible for the movement
of the planets but there is no
responsibility to get there and some
people try to describe what else we need
and i wanted to ask you what you think
is required so the absence of
responsibility
and and something else what what is it
that creates responsibility gaps so
that's about the definition
yeah and the second uh part of the
question will be about the significance
of those gaps i mean it strikes me that
hardly anyone makes explicit why
responsibility gaps in particular the
backward looking negative ones are
problematic so you see the just war
theorists
who say well responsibility is in some
way a condition of for just war
but in other contexts it's kind of
puzzling why we should we
should worry so much about it couldn't
you just say well
being responsible for bad things is a
burden and if we can move on to an
insurance scheme and get rid of the
responsibility stuff
wouldn't that be a good thing so why
should we worry so much about it
okay great yeah great questions let me
start with the first i mean so
yes i agree with you of course that's
the absence of someone who
is or could to be held responsible
that's not enough there has to be
something more and i think there are two
main things
uh that could that could happen that
would make it seem as if there's a
responsibility gap
the first one is that it seems that it
would be right that someone should be
held responsible
uh because the occurrence in question
may be
unlike the movement of the planets or
like a you know gust of wind or
something like that
if we have technologies or even we have
groups of people interacting in some way
there seems to be some sort of agency
there and uh this
whatever happened didn't happen sort of
totally accidentally
and so it just seems maybe seems right
to us into it to be speaking so to speak
that someone should be held responsible
it may also seem good that someone
should be held responsible it would be
better
uh and people in general find it
you know extra bad sometimes if
something happens
and no one can be blamed i mean so this
may to some people this is
i mean especially when it comes to
punishment some people don't like this
idea
uh they think that we may have kind of a
desire to to punish
sometimes and so we feel that someone
should be punished and it would be good
if someone would would be punished for
something
but actually would be better to try to
get rid of these
kind of desires i mean so uh steven um
kaya felt us on a paper that we tried to
debunk the idea of responsibility gaps
because the argus that often it's driven
by a kind of desire for punishment and
that we should try to rid ourselves of
but uh it can also be on the good side
you know we
something good happens and we think it
would be nice if someone could be
you know praised or you know rewarded uh
but we
might not be able to find someone who
deserves it so
in general then uh it seems right or it
seems good or maybe we just want
someone to be responsible but we can't
find anyone
who could justifiably be held
responsible at least
to the right degree now uh and then the
question was maybe we should
go the uh sort of stephen curry felt uh
route and try to
uh you know accept that people are not
responsible
i mean i um yeah i think
uh sometimes it's just psychologically
hard because people have a kind of
strong
disposition to want to find people to
hold responsible
uh it can also be a way of achieving
more you know control and meaningful
human control is you know one of the
topics of this particular series
so the sense that things happen in the
world beyond anyone's control and no
one's responsible
uh i mean i i personally think and i
discussed this in some other work in
progress that people sort of
actually value control and they've had
they want
things to be happening in a responsible
way that we can you know say that
you know someone can get that be praised
or
blamed for and it's uh you know it seems
bad to a lot of people when things are
out of human control but
but yeah so with individual cases i
think we can always discuss would it
actually be better to not
try to hold people responsible but
that's a bigger
discussion yeah i think you're right
about that if i just can
say one more sentence i guess so i agree
with what you say but i think these
answers also show that we are still
unclear about the value of moral
responsibility in different
senses what what is the value that
people have moral responsibility for
certain things
and there are some very i think
fundamental discussions in philosophy
but in
the in a ethics it seems to me that we
still don't really understand that
but yeah but thanks very much friends
great no i agree it's a bigger
discussion and it's uh
uh as i said i think it has something to
do with the kind of
value that people implicitly put on the
idea of control but uh
that can also sometimes uh go too far
and sometimes we maybe should give up
some control and
try to be less controlling but i still
think that people have a desire for
control over certain parts of their
lives and responsibility has to do with
that
okay thanks uh so next in line is
ilsa
yeah um i basically wrote my question in
the chat
uh saying that if you look more
fine-grained into
backward looking and forward
responsibility you can see that it
consists of several elements so i was
wondering if you also take this into
consideration in your
classification
uh yeah absolutely so i mean that can
that question can be taken in different
ways and so for example there's a paper
by
daniel tigard where he discusses
what uh gary watson and david schumacher
call the different faces of
responsibility
so they talk about attributability
answerability
uh accountability um for example and and
then
uh tiger says well you know you can also
i guess formulate
accountability answerability and uh what
was the other one attributability gaps
uh and then there could also so that's
you know in terms of what exactly do we
want do we want
to find someone to hold to account do i
want someone to provide answers
do we want someone to say that that was
the person who did it
um that's one thing and others would be
you know what exactly are the criteria
has
to do with control with agency uh with
resources etc so yeah so i think within
the broader classes that i talked about
you know you can have many different
things and
a lot of what people have discussed
would be would still fall
under the class of what i'm calling
backward looking negative responsibility
gaps
but some of the things fall into the
other classes and
again i want to say that the whole idea
of value alignment seems to fall into
this
otherwise um seemingly unoccupied a
class of
forward-looking positive responsibility
gaps
but yeah this this is for a higher order
discussion of
broad classes and i agree with you hill
so that
within the classes there's a lot of
further discussion and you can sort of
sub
classify within the broad classes that i
presented
okay thank you okay
and next question is andrea but on the
connection with free will
hi sven thank you so much for this great
presentation
so my question is about free will as we
all know free will is traditionally
considered crucial
element for moral agency and moral
responsibility
but recently i did an empirical study
and i was surprised to see that the
large majority of the people that
responded to the survey
actually did not consider free will as
relevant for moral agency in the context
of artificial moral agency
uh could you provide your opinion on
this
yeah um yeah that's an interesting
question i mean i
myself think that agency is actually a
very complex
uh concept and so uh
with many different parts to it and so
an agent is able to
you know perform actions to plan things
to talk with other agents to be
held morally responsible maybe has free
will or doesn't etc
is able to think about sequences of
actions and make plans for the future
to reflect on the past etc so i think
these are all examples of things that we
you know associate with what
philosophers call agency
and uh because it's such a
multi-dimensional concept
i think that that means that there could
be different kinds of agents
and so i do think that a lot of people
think that when it comes to human agents
that we have
what we call free will of course people
mean different things by that
expression uh but maybe when we think
about
other agents uh and let's say an insect
a
house fly or a wasp or something like
that or be
uh does it have free will well maybe of
some some other sort but it does seem to
perform
actions of some sort uh and in the same
way
um you know some ai system or
self-driving car can either go
left or right you know it comes to a
fork in the road it's very hard not to
think in terms of it decided to go left
let's say
but you know does it have free will
probably not
uh but a lot of the things that we
associate with agency could still be
true i mean it's
reacting to the environment and the
seemingly intelligent
way that seems to be in the service of a
certain goal to get to the destination
because if you had gone right then it
would you it would have been a detour
and it would have taken longer etc so
there seems to be some sort of
uh agency there but yeah free will
that's maybe not
seem to be there of course i mean again
free will is also one of these concepts
that
i think not only can it be interpreted
in different ways
even within a certain interpretation
it's probably going to have lots of
different aspects to it
and i don't know why some of those
aspects couldn't be realized in
a sort of an advanced ai system uh even
if maybe
something like being conscious of what
you're doing which may i think we
associate with free will
uh you probably wouldn't want to say
that the ai system or robot is conscious
well but then again i mean consciousness
too
has aspect that you may be able to
replicate in a technology so
i think that these are all just it's not
just like either something has agency or
consciousness
or free will or not i think it's usually
better to say
uh these things have different parts to
them and uh
technologists may be able to kind of
click click some of the boxes
not all of them and have something like
kind of agency
kind of quasi free will kind of some
sort of
beginning of consciousness and it's not
really an on off matter uh
although when we are asked you know
could uh you know the
ai system have consciousness we might
probably gonna say no
does that free will we program to say no
but i think when we sit down and think
about it we should
be more nuanced and think that okay it
partly has to partly fulfill some of the
conditions that that's the the sort of
the take i have on this thank you so
much
great uh acadi
yes uh thank you sam that was a great
talk fascinating i like this angle on
responsibility on responsibility gaps in
terms of uh positive uh
kind of effects but then i wonder if you
see any potential
let's call them side effects of
emphasizing the need to address
positive responsibility gaps and then
what the
the specific context that comes to mind
is uh that it might be tempting for
uh some people call them ai optimists to
appeal to this kind of reasoning
when advocating for their technological
ai based solutions to
the world's problems so do you see any
parallels there
and if so yeah how can we proactively
address that
yeah okay so yeah thanks for that that's
a great question uh
yeah i i in a way i don't like my my
my conclusions because uh i'm sort of
saying that there's
it's hard to find someone to hold
responsible for creating good outcomes
involving ai
and the reason for that is that uh both
ordinary ways of thinking and people's
attitudes and a lot of ethical theory
are just focusing much more on negative
responsibilities
but we desperately sort of need to make
sure that people create
a good rather than bad ai systems
and we we are aware of the risks uh that
uh you know could are involved so that
that's why i said it towards the end
maybe we can sort of try to rebrand some
of the value alignment into value you
know to ai safety a securities it's
really a matter of
avoiding bad uh consequences rather than
positively bringing about value
alignment and that is more treated as a
kind of
a means rather than the end but on the
other hand
that's also not satisfying because you
know the reason that you would want
advanced ai systems would be uh not as a
means to something else but
as a goal i mean you you think that uh
you know ai systems that could detect
cancer i mean in the example that i
think i mentioned
and also that's scott robbins one of
your former colleagues during delft
discusses in a really interesting paper
uh from 2019
mines and machines uh i mean
we want this good outcomes and so it
isn't just a means for avoiding bad
outcomes
involving ai and if if that was only the
problem why would we want the systems in
the first place so yeah i um
i don't kind of like my conclusions but
i just think that there are
implications of the way that
philosophers usually talk about
the difference between positive and
negative responsibility
and also the way that people in general
just they feel that you know
they're they shouldn't be held
responsible for bringing about good
outcomes for other people
they should be able to focus on
themselves maybe their children their
friends and family
and so long as they don't harm other
people and this is what
sometimes called well what people talk
about that's meals harm principle
uh you know to do whatever you want as
long as you don't harm
other people um and but it might seem
well you know if people are creating the
systems they should be striving towards
the good but if there's no
responsibility there yeah so i mean i
don't really have anything good to say
other than that i don't
like my own conclusions but there just
seemed to be implications of
the way that philosophers have discussed
some of these topics and if you just put
them together
you get what i was talking about i think
great thanks okay for now the last
question on my list is
art
yes uh thanks for your talk sven i'm
writing my master thesis about the
responsibility gap
in a forward-looking way so it really
provides some useful insights so for my
thesis
and i was wondering i also read about
the problem of many hands from the pool
and he provides us with two solutions to
alleviate the gap
and the first one was virtue and duty
based so
that sounds more like that the value
aligned solution that you also provided
but he also mentioned that we could use
some procedural
approach to distribute more
responsibility for example with the
procedures of john rawls so i was
thinking is that also
a possible way to deal with the positive
forward-looking moral responsibility
yeah um uh i mean probably
i i'm not super i'm familiar with the
the virtue idea that
uh thunder paul also discusses not some
very familiar with
how to apply the roles and i idea to
this particular case but i mean that
does seem
promising i mean something that's maybe
a little bit related that i've been
toying with myself
is to think of responsibility i mean as
i said
if you're positively held responsibility
after something good has happened then
you know it's a benefit to you to be
held responsible and some people even
think that a negative responsibility can
be a kind of benefit because it shows
that you're an important player
i mean there's a kind of interesting
case the norwegian mass murder on this
brave
was first deemed to be insane and not
responsible for his mass murder but he
actually
wanted to be insane and to be held
responsible
because he wanted to be you know true uh
i don't know fighter for his cause and
if it was didn't
insane he couldn't um you know be viewed
in that way so he
preferred being sent to prison uh and
held responsible
to being sent to you know to care and be
regarded as insane uh and so even with
negative responsibility people might see
that as a kind of a positive thing
uh but in general i think we can think
about okay who is it that would
benefit for example from ai technologies
maybe
uh the reason that because they benefit
they should maybe also
be more responsible for people who maybe
benefit less
and so we can use a kind of fairness
reasoning so the more you benefit from
some technology some practice the more
maybe responsibility you should take on
for any outcomes good or bad for of that
practice or that
technology uh i think that would
probably be
in line to some extent with roles in
reasoning and you can probably provide a
kind of roles and account
of why that would be just fair so
yeah i see potential there but uh
although i have started thinking about
this a little bit myself i haven't
looked at
specifically the rolex in way of looking
at it
yeah okay thanks yeah i was not really
specifically about the roles in a way
but more
and the fact that you can deliver some
procedures beforehand and think of
scenarios
in order to what can happen and then uh
distribute the responsibility beforehand
sounded like a different way of
approaching it than based on the duties
so
uh yeah right yeah no indeed and then
you can ask
okay so if someone wants to you know to
do something in some domain or develop
some technology introduce some
technology
and with potentially then benefiting
themselves
you know for any uh income that they
might earn etc
then perhaps the procedure should be
that yes then they also have to take on
responsibility for anything
bad or good that might happen so they
they could
you know get the rewards that they may
create for the value that they
recreate but they should maybe also then
internalize
any and any of the costs or possible
problems that
so that might be fair in a sense so yeah
i i think that's an interesting
way of thinking about things okay thanks
okay the new question uh luciano yeah
so yeah thank you very much it was
really interesting and challenging to
think about these different angles on
responsibility gaps and i really like
the way you connected this to
value alignment for example this
tracking condition to be alignment i see
a lot of parallels was very nice
my question goes into this when creating
more and more
ai systems things we have also
what uh mark cuckerberg was talking
about the problem of many things
we have the problem of many hands as
well the human things but we also have
another side more
different layers of ai agents that might
interact with each other make this kind
of
collaborative decision so how do you see
the problem of many things
how how in which of the of the boxes
would that have an impact and how do you
see this
the seriousness of it i mean i i think
it's a
very serious problem i re i really do
worry a lot about the many hands and
things
problem here especially with ai systems
using a lot of data and uh
you know different companies producing
different parts of the system and
you know different companies providing
the data etc so
there are just so many hands and things
and so many much data
that it's almost impossible to not worry
about a lot of responsibility gaps
uh so yeah i mean i think this is gonna
create problems in all of my boxes
basically uh
because i mean it's it's quite different
than some more
old-school technology and there is maybe
one producer of the technology and then
one user and it's all kind of isolated
but with with ai it's so spread out
and so so many hands and so many things
as
you know as mark cochlear was putting at
that but yeah i i'm
worried and i think all boxes are
implicated
i completely agree with you and i think
in my
own earlier work i was more uh you know
optimistic about the
possibility of solving responsibility
gap problems uh but then hadn't maybe
thought enough about uh this problem
many hands and things and
you know how many different players and
data sets and all sorts of things are
implicated
yeah great so if i may follow up on this
herman do it
can i have time okay okay uh
so you you also on when concluding your
talk you mentioned about like that this
this approaches including your own the
one from
from santoni and some others they are of
course not a checkbox
like well what is it all is everything
so if i check all the all these boxes i
pick everything then there's no
responsibility gap
so it's not like this of course and but
do you think that we can
ever get there to get this a list of
check boxes
or or do you see is this context
dependent is this
a goal to to strike through according to
your vision
yeah i mean uh in a way i think it's
it is a goal insofar as people really
like it if they can
distribute responsibility to to people
because we we feel more comfortable when
someone is responsible for something uh
and uh
i mean one and what i had done in my
previous
uh writings about this boss in a way to
provide a kind of check
checklist of like if you take most most
of these boxes then you're probably
responsible
the problem though is that uh and this
was something i kind of briefly
mentioned and then someone wrote a whole
paper uh rose the young and that
kind of calling me out on this that you
can have different people taking the
boxes on the list
and so maybe someone understands the
system
another person is the person whose
interests are actually tracked
yet another person is the one that's
kind of monitoring the system in real
time
another person or set of persons uh you
know has some other relations
so you you get again the problem many
hands coming in from kind of another
angle
if even if you have a checklist because
different people may
then tick the boxes uh one solution then
would be to say that actually
what people are responsible for are much
smaller things than the overall
outcome uh maybe someone is responsible
for you know one aspect of the outcome
some
one else is responsible for another but
that's also a problem because
at least it's not satisfying because
people in general tend to like it when
responsibility really
is kind of local localized so there's
one agent or maybe group of agents
who's most clearly responsible rather
than having responsibility be kind of
watered down and spread out across
many people if everyone is responsible
no one is responsible i think it's that
kind of attitude that we have
so it would maybe be desirable but it's
it is hard in practice yeah indeed the
completely agree also this diffusion of
responsibility through other layers is
complete
very complex so thanks again sven thanks
really interesting
great so i don't think there are any
questions so
no i can ask my own question finally
um so when you described responsibility
gaps you did that
in epistemic terms so that we don't know
who is responsible
uh where in the case where it seems
appropriate that someone is responsible
so is that your is that do you think
that's the main kind of worry that we
have or should have because you can also
think uh
responsibility gaps in terms of in a
metaphysical term so that
there should be someone responsible but
no one is responsible or more immoral
terms that
not the right person or not the right
collective
is responsible so the distribution
should should change
so do you see that as three different
and relevant
problems or do you think one is much
more interesting than the other
so what's your what's your take on that
yeah no no okay that's a
great point i mean so in the written
version i talk about what i call
real and apparent responsibility gaps
and so
i guess a parent would be the uh
epistemic so you know
we don't we think there someone should
be responsible we don't know who is and
so on
whereas the real would be that actually
there is no one who who could
justifiably or rightly being held
responsible
and so we it could of course also happen
that we do hold someone responsible but
they don't really kind of deserve it so
to speak
especially when it comes to as i said
people are going to be
quite willing to step forward and take
credit for good outcomes i mean whether
it's produced by an ai system or by a
person
we're always eager to take credit for
good outcomes and so that
can mean that we maybe think that
someone deserves
praise but really uh from a kind of i
don't know
metaphysical point of view they don't
really so yeah i would agree with you
that we should
uh this i mean this would be yet another
way of getting more complexity to the
uh i was trying to keep things simple
but yes you're right
uh and uh yeah the language i have been
using
in my written work was in terms of
apparent and real responsibility gaps
uh perhaps it would be clear to talk in
terms of epidemic or metaphysical or and
so on so i agree with you that this is
another issue
that we should be thinking about
absolutely okay great
yeah yeah so yeah so you you of course
introduced four distinctions uh
everyone has his own distinctions uh
that the uh she likes
um but um yes so so did thinking that
you called it real and apparent
so so that's at least
how i interpret it seems to be a
different thing so sometimes there
seems to be a responsibility gap but in
fact uh
it was just an accident for example and
if we
look closer in fact no one is and no one
should be responsible
so and a self-driving car uh
causes and that causes an incident but
it is yeah but
when we look at it it is an accident in
a moral sense
so uh human beings can also be involved
in accidents and
then also no one is responsible so is
that the kind of thing you're talking
about or
is this again a different distinction
well
i actually think that's a really nice
different distinction so i think
you could subdivide the apparent
responsibility gaps
into ones where uh yeah you're right it
actually was an
accident in the sense that we know no
one should be held responsible but
uh maybe we think someone should be but
we don't know who should be
responsible and yeah i can't remember
exactly how you just
made it the distinction but it's as you
were speaking it
seemed to me that you could candidate
take both of those and say that those
are different ways in which there can be
an apparent but not
real or actual responsibility gap so
yeah
there's no sort of end to the uh the
possibilities in terms of how to sub
i mean i i actually think there's a
value in making a kind of a taxonomy of
all of these different ways in which
responsibility gaps can occur because uh
that just helps to show how
sort of problematic this is and how many
different reasons and
types there can be a responsibility gaps
but yeah what you were talking about
just just now
seem to mean to be two kinds of reasons
why there can be apparent responsibility
gaps
okay thanks
um so it's two o'clock um so
um our official time is up um
i i don't know if you have anything so
we could
people who still have time can remain in
this session if you have time sweat but
if you have to go that's also fine
uh so let me first ask you do you have
some time for some more questions if
there are any questions or
i know you're busy so that would yeah
well first of all uh
uh before people leave let me just uh
say thanks a lot for those who are here
and for your very nice questions and
comments i mean this is
new material and i'm very eager to
discuss with people so very very much
appreciate
the opportunity to talk with you and i
would certainly be
willing to hang around a bit longer if
people have more things they want to
discuss with me
i would be eager to hear any further
comments or questions and also
you can always email comments and
questions to me as well
it's kind of crazy enjoy listening to
you
thanks great thanks everyone thanks uh
sven
and um yeah virtual or an actual
applause
thanks thanks so much so uh
yeah so i have a another question but
so if there are other other people who
have a question please
shout out
i i will stop the recording now that