Human control over fully autonomous systems: a philosophical exploration (Giulio Mecacci)

turns as usual we're probably going to
leave this room potentially even more
confused than we arrived in some cases
in some cases i'm not sure this is one
of those it's a good thing right but
we'll see so let's get it started um
there is a lot of talking about whether
more automation and artificial
intelligence
will bring about less and less human
control
right by simple definition
more automation means less control
but of course on the one hand automation
is highly desirable right it promotes
productivity it reduces human effort
and also when it's used correctly uh it
improves the quality of
operations of what we do um and on the
other hand
however we would like to ideally be
able to remain in control of this
autonomy
and this is what i call here the control
dilemma
right um but first things first
why do we want or need to remain in
control of these intelligent
autonomous technology
there's at least two big families
let's say of preoccupations so the first
family of concerns regards
safety this ranges from safe operability
for instance when we deploy systems we
don't know very well
or are that are unpredictable to some
extent
and a two existential risks that's
that's the other
that's the other problem and the whole
debate about super intelligence if
you're
if you're familiar with that we've seen
many important authors recently
dedicating some serious efforts to this
um stuart russell for instance very
recently his last book human compatible
hey aai and the problem of control
i'm talking about these risks for ai
but there's a second family of concerns
which regards
moral responsibility so it's something
like who's going to be blamed for the
actions that are initiated by an
autonomous system
many have um observed for instance that
artificial intelligence
especially with regard to certain deep
learning techniques
has a transparency problem in the sense
that we cannot easily find the reasons
why
certain decisions have been taken from
the
artificial system and this makes it
really
hard to trace back those auctions to
some human person that's accountable for
them
and also in many cases even if these
persons or
person are retrievable we have troubles
genuinely blaming them
right either because they didn't do
anything intentionally
or maybe because they couldn't foresee
actually what would have happened
and so on and and other reasons like
this
so it seems to be important for many
reasons to remain in control
but we might just not be in luck
maybe we have to to give up on that at
some point
since since we're going towards like
increasing and increasing autonomy right
so many have tried to provide solutions
or maybe i should say sometimes work
arounds
to minimize the issue of giving up human
control
especially for what concerns the problem
of responsibility
so just just here a couple of very
oversimplified
examples here we have for instance those
who propose that
something like nobody gets uh blamed
okay
nobody would just stipulate some sort of
agreement
and where we specify who's going to pay
legally speaking and we settle for that
but this approach
um let me only say that it has been
criticized
also by myself by for instance
highlighting the importance of some
forms of
moral responsibility that should never
be dismissed
like this i'm not going to go into
detail here because i want to get to the
point and i never do
but as a i've been there in other talks
and papers on this and there's actually
a forthcoming paper
where uh phillipos antonio de cr and i
would discuss this thing among other
things
so hope you stay tuned if you're
interested in that but there's there's
them let me go to the other
point there are others like some of my
japanese colleagues
roboticists who genuinely give their
best
shot at making artificial autonomous
agents
responsible for their actions uh
so and they investigate forms of legal
personhood um this approach
has frequently encountered criticisms in
the form
of you know there's something off with
kicking my refrigerator because my beer
isn't cold enough you know how it goes
it's like this uh but of course i'm
to to a large extent sympathetic with
this uh attitude to to a large extent
not completely but
it's worth considering of course that
artificial autonomous
agents might sometimes even soon even
soon become very similar to us humans
and one might want to be prepared i
think that's what moves this
approach to the central idea that
encourages this approach
but all you know many believe and we
good with good reasons that there's no
way
to have the cake and eat it
right so in a sentence to maintain
any sort of human control
in the meaningful sense over highly
uh but well let alone fully autonomous
system is impossible it's it's
impossible just impossible
so if we can control them it means maybe
that they might not be autonomous enough
but in this little talk here i would
like to explore like a few philosophical
ideas
that come from the tradition and
philosophy of free will mind and action
including a very recent idea of
meaningful human control
to see if there's anything that might
let us have half the cake right we have
the cake
and at least we take a little bite of it
i don't know maybe we can settle for
that
and we'll see um some of you may know
and it's been uh
mentioned here already for the past few
years i've been working at deepening and
operationalizing this philosophical
theory
of meaningful human control over
autonomous systems
driving systems where was the use case
but the theory itself is neutral to the
case and and that was developed by
filippo santorini this
in a is paper a few years ago and this
theory this is important
this theory was really devised with
overcoming this dilemma between
control and autonomy in mind among other
things but but this is what i
really want to take uh out of of out of
this this idea of meaningful human
control of this theory
uh i know that some of the crowd here
might think i'm taking them like at
nauseam
they're allowed to switch and switch off
the audio the next few minutes
but i would recommend staying anyways
and so and wait for them
for the twists theory
the theory of meaningful human control
is theory sees two conditions
right for control over autonomous
systems they're called tracking and
tracing
okay so the degree of human control of
the meaningful kind
uh would depend on the degree these two
conditions are satisfied okay so
um
the the tracking condition to the left
sets a specific requirement a property
that these autonomous systems should
always display
if we want them to be controllable by by
human agents by humans by us
persons this property is that these
systems potentially even complex
socio-technical systems you know
operators devices the infrastructures
these systems um
really um should should display this
this property of
um covariating their behavior
covariance their behavior you see the
cogs there
with the relevant reasons of the
relevant human agents for for doing
things
or not doing these things
with their intentions in a way they will
and the second condition then we'll go
back to this one actually we'll focus on
this
um the second condition is called
tracing and it requires several
competencies from a user of the system
and that means making their it aims at
making their involvement
in controlling the system as relevant
as possible right there should be a
person it says at some point of the
design or use context
of an autonomous system that has a
special awareness
of its functioning and an awareness of
the moral
role that they play in that system and
this would allow as the word world
tracing uh suggests
to trace the auctions of of a
controlled system as effectively
as possible back
to one or more human persons humans that
were put in charge
so trace back to them the options of
these systems
now what you should notice at this point
is
that these two conditions they aim at
two different aspects of control right
they have to be slightly different if
you will the complementary scopes
the latter condition tracing aims at
facilitating
uh the possibility to attribute
responsibility in a way that is fair and
just as possible
but it is less less concerned with
the uh the nature of the actual
interaction between
controllers and the controlled system so
if we would take tracing alone as as a
condition for control
so the whole meaningful human control
theory would be a
normative theory we've seen several
normative theory about responsibility
and accountability
but it wouldn't tell us much about
control in the other sense control
itself any other sensory
um about whatever connects
controllers and control systems
so the tracking condition the first one
is there for this reason
and it requires the system's behavior
to cover with these certain reasons
to act but what what's up
what are these reasons you you ask of
course why
reasons are not auctions for instance so
normally you'd want to control lowly
autonomous systems with
some sort of control panel you want a
system to behave
according to the buttons you push that's
that that's behaving according to
auctions
reacting to options not however with
highly
or fully autonomous systems and one of
the reasons
being that we're very bad for instance
that's supervising and vetoing
so reasons to act is a generic way to
call
human intentions dispositions goals
plans and even values we have argued
the idea is that that assistance
the systems assistance behavior
simple alignment with those reasons
would grant the space for higher
or high degree of autonomy while
maintaining also the right degree of
moral involvement of control in a way
and therefore
of course yeah it's more control
the whole idea is it wasn't invented uh
as uh like it wasn't a fresh start it
comes from
philosophy of mind and free will in
general let's say
a general intuition uh there we have
been trying for centuries
to sort of understand how mental things
like intentions of reasons are connected
to physical things
like actions and in particular very
important free actions
so we're almost there not yet you give
give
us philosophers another thousand years
maybe two thousand
um and we'll we'll see what happens
but now here's the issue with this
here's the issue here's the issue
i said we need autonomous systems
behavior to co-vary
right with human reasons so there needs
to be some interaction
going on right um so these reasons
have to somehow cause the system's
behavior
they need to steer it to keep it on
track with our ever changing
moods or the uh ever
change value changing society
our changing needs yes
and no mostly no there's
there's good reasons why this condition
doesn't use words like causing
or similar words so reasons reasons
the reasons are well known
to be somewhat problematic causes
for action so reasons are not good
causes for auctions
those reasons that should steer push to
which the system should respond to
right so not good causes i
um donald davidson amongst other
philosophers
he discussed this problem at length in
the context of of the mind body
problem to oversimplify
it might be it might be hard to retrieve
a strict and well-defined
law like a physical law right which
binds the reasons
for an action and the action itself
this is for some at least some due
mainly
to the fact that reasons are high level
descriptors
in a certain psychological language and
this language seems to be hardly
reducible
to its physical counterpart which is of
course expressed in a different
formal language the language of physics
mathematics
maybe so in philosophy of mind many
would settle for this would say after
all physical events
okay actually control wants action
that's not what we're denying
they're just too complex and fuzzy to be
described
effectively in in in physical terms
so so we use reasons to describe
those uh those events
those from physical phenomena so we use
more abstract explanations in a way
which we can handle
much better and in some sense so still
in some loose sense we can say still
that reasons cause
options right but we have a different
problem here
because we have to design
behavior and not only to
explain it so so reasons are very good
explanations
explanation for auctions but
but we have a we have to take a
different more aware perspective
from a step before design
right so we need to understand that the
nature of this link between
reasons and autonomous systems behavior
and the tracking condition here it just
says
um that human reasons and the actions of
these systems
should go hand in hand should be aligned
it doesn't say whether this is the case
because they just always let's say
happily agree
or because they talk to each other
so
how do we solve this um kind of
principled problem
so one idea would be to find a way to
establish
this link that we're missing between a
controller's reasons to act
and the behavior of these autonomous
systems but then one could think
we have said reasons might be right in
some hardly accessible sense in the
brain and
therefore in some way there are causes
of an option
so one way would be one way to establish
this missing link would be
uh through some sort of i don't know
brain reading device
that's capable of classifying mental
states
with interpreting abstract intentions
putting them into action um
okay but um i think this this is a
little
misleading and i have some concerns
about this idea
ranging from well technical visibility
to its
principal soundness let's say i'm going
to be very
very superficial here i apologize
because this sort of deserves a whole
talk
maybe longer one than this but first of
all from the technical
side we're far away from having a
functional
neural imaging system an ai classifier
that is sensitive enough
to discern the adequate nuances in
thoughts and intentions so
achieving such technology might require
extremely high
resolutions ai classifiers that are very
well
smart they're sensitive and as specific
as
persons and understanding thoughts i've
defended this point
in the paper some years ago but this
means
very broad and very intelligent
algorithms or neural networks huh
i'm not saying this is impossible i'm
not saying this but i'm just wondering
what do we do
while we wait for the technology to be
at the at that point to be invented
and the second concern is also sort of
is maybe
more more deep uh somewhat more
principled it's about the possible
inherent difference between a reason
on one end and the neural event on the
other hand
so we should be i believe very careful
not to
trivialize the notion of reason
uh in fact these reasons as i said
before
include include different different kind
of
entities
like moral goals motives
and even values so it's to these
abstract
entities that we want a sufficiently
free and autonomous system to be
sensitive to respond to to align
to so the extent to which these things
are in
retrievable in someone's mind let alone
in some interpretable
neural event is very dubious
so these these entities these weird
entities these reasons
they extend over time they emerge at the
level of society
and so they're they might just not be in
somebody's or more person's head right
it's
not what they are maybe so being short
of ideas
how to make it how to make this work
this connection work
this connection between human reasons
and systems uh
actions i i thought i went back in time
meeting uh gottfried leibniz
uh well i thought you know maybe
i'm reminding of something so as many
philosophers
of of his time he was trying to make a
lot of things work
many different fields doing many things
and one of those was the usual of course
um
working out the relation between our
soul or mind
and our body and therefore
actions our options so one of his ideas
was
that actually causality might not have
been necessary
might be something necessary he thought
god is a very good clock maker
and they designed all things
so well that they would forever
work in harmony a
pre-established harmony
so any causal interaction between two
things
would be mere illusion
and these basic entities the world is
made of
he calls them monads that's why the
monodology of
technology here uh called the monads are
these just self-sufficient systems that
god pre-programmed to harmonize
with each other like perfect locks they
they sort of all run together but they
never communicate
with each other or with the designer
and since hey i would really like to uh
play god
i thought this would have been an
interesting analogy but yeah in a way
in a way in a way we do right
so if i keep playing along the lines of
this analogy
my question becomes should we consider
at least as like an option to
investigate the possibility to conceive
control
without any such causal connection
between controller
and control system which is at least in
a big part already already contained
in the in the idea of tracking and
meaningful human control
but of course like we have to make
exception of this initial design
phase there they're just the contact
right when we as uh like it was for the
late night
leibniz and god we set things in motion
and what does it mean to design for this
kind of
control so i thought another metaphor
came to mind uh and uh
and let me mention it it's a silly one
but not so silly the train
right so in a way the train goes where
we want
all right so does does we want
usually but it doesn't require any
additional input to steer
it doesn't require us to constantly
intervene
to sort of keep it on track
with our reasons um the tracks
the railroad allows the train to be
going where we want the whole time it
travels
and there are design design of these
tracks it expresses a
very dense history of societal values
and moral reasons
so they tell a story about for instance
politics
and economy and about the people's good
reasons
to meet each other and stay connected
moral reasons too so i understand this
is not a good example of sort of
intelligence
of or flexibility or autonomy yeah
even but i in some sense it might be
but it might also be a good example of
control so
should we consider maybe more
intelligent
and more autonomous systems
like trains with very many of those
tracks
this is it can this inspire a little
could we design for instance all of them
to go where we want so we can we
set them up in a way that that
they won't fail us but they'll be
flexible enough
for us changing our minds we design
those tracks and this is harder
so to go where we might want to go in
the future
in this value changing society but to
never go where we should normatively
never want to go
so final question is really if this is
even
something would this be a sufficient for
us form of control would this be a
sufficient form of autonomy as it was
for
for leiden it's or would this just be
another
over complicated way to sort of to give
up
anyways on control autonomy or both
i'm gonna i'm gonna leave you with this
thank you so much
because i feel like this becomes
a field for designers more than
philosophers
and i would love to sort of see if if
any intuition or or have i stimulated
any thought about it thank you so much
and thanks
thanks like this great
julia thank you very much extremely
fascinating talking i really like it
this uh yeah someone have question you
can just yeah just
raise your hand or just say something or
just vote on a chat
or anything right not only questions or
answers to judy
questions that's what i want if i may
thank you very much for presentation um
i'll turn on my camera as well otherwise
yeah just looking at a screen for us
hello so let me put you in the right
spot
yes now i'm looking at you and seeing
you back um
i i that second idea i was it got me
thinking about what is ai to you
um because i would actually be quite
fine with this idea of perfect design
being the solution for my trains or my
dishwashers and other systems but i
might be less fine with it when i'm
thinking about for example my kids
and i have a feeling that i will be
somewhere in between that spectrum
uh so where would you place ai and how
do you think that might relate
to this idea of perfect design will be
enough
so these uh thanks so much thanks rob
very very fascinating this brings me
back to so many things and discussions
and
free will so let me let me give you sort
of the counterpart
that what it's what we do in in
philosophy free will right
there's this whole discussion about
whether
determinism a world that is
that is made of uh physical laws that
have
one way to go right it's a chain of
adventure there's no way to do
otherwise in physical terms
right and then when we think about
ourselves as
much as so your our kids right or
ourselves
and when we think about when we think
about ai we're made of the same stuff
we're made of the same thing we're very
similar
right so the way not the answer but sort
of
the observation for your question is
clearly
um if we're happy with us being
sufficiently free
and we're in a way even if we're not
sort of intelligently designed but but
at some point we're made in a certain
way
uh because of evolution and because of
so we have our own constraints and
things we can do
things we cannot do our body is
made in a certain way we're trying we're
doing
we're overcoming these limitations daily
with technology that's one of the great
things that technology does
but we have for instance are the limits
and challenges of our cognition that are
there
again trying to overcome them but if
we're happy with
us have being constrained we might be as
happy uh with with ai
and then the the next question that you
made
actually is but then are we can
we define ourselves as being under
control of evolution
for instance that's something that sort
of
because who designed us right now we
design we want control
through non-intervention through just
just that
moment in time at the beginning when we
design this technology in such a way and
we want to call that we want to see if
we can call that control and are we
under control of our own nature this is
spinoza by the way i believe
i believe i don't know if there's
philosophers who want to kill me
in this uh in this uh audience
they're welcome because i'm this but
yeah forget about it um anyway
yeah this is this would be what i
observe out of your question which i
find very fascinating
i cannot hear uh we have a question from
enote on the chat uh yeah
you know do you wanna yeah sure so julia
thank you very much
i'm i'm i i sort of tripped over the
last
remark that you made about are we under
control of evolution
so when you say so before i go to the
question in the chat
and i'm sure you can answer this one but
when we say control do we always
assume that there is a controlling
entity or can it also be something that
is emerging
i think it's it's emerging i think it
should
we should consider that it's emerging
ideally i'm sorry why do we call that
control
influence but that is the challenge
so meaningful human control really is a
way in a way
to smoothen to to say it's it's more
like an influence when you when you
think about
at least when i think about our
theory of meaningful human control i
don't think
about that kind of control that is
direct operational but is more like an
influence would that still be within the
boundaries of control
that's your question which is which is
very good of course what you're saying
is
but why do we call that control is that
influence
and where does it where does influence
finish
and control starts right do we want to
define meaningful human control as a
softer form of control do we do we
associate control with a sentient being
do you mean the controller do you
controller yes yes so the
the the person the phenomenon the
whatever
whatever that exercises that control
whether it's whether it's controller
influence and that was not the intention
of my question but i'd see your point
but now i'm on that question of uh you
know
deliberate sentient etc i mean
i i mean i'm a big fan of a selfish gene
and those kind of
books which i don't think talk about
control
but yeah that in a way that book it
represents a little
bit in our terms like translated in our
terms
we've been many times claimed to be the
control
that is exercised by
societal values by the
system at large that's made of
a lot of sentient beings but also
uh made of of these these regulations
values intentions ideas are those
those values that maybe we can conceive
as controlling the direction
of the technology where the technology
goes
are those sentient in a sense or do they
emerge you you use the merger i like
that word very much
okay let me go let me go to the question
i put in the chats
so this discussion about the reasons and
actions etc of course is something that
uh
humanity has always you know been
subjected to in a way right if you go
out and get food or if somebody attacks
you and you pick up a stick and you kill
someone or defend yourself
there is this action reason um causality
etc
but in the past i don't know 50 years
100 years or so suddenly of course we've
made that step to
interaction with technological artifacts
and ai being the
the most recent one at a level that we
can hardly understand so i was wondering
if we discussed the
interaction between reasoning and
actions let's say in
human history versus where we are now do
you think they have the same answer
yeah i'm not sure if i make my question
i'm not sure if i'm not even sure if
if i phrase it well but
so do you do you see where i'm trying to
go or or am i
did i lose you so no then it's not
because of you i think i think
i'm just trying to digest the depth of
this question
um so the interaction between
rationality rational entities and
and actions and the world outside
between mind
and body okay did it change
when we discovered
that we could design intelligent beings
or or what you mean maybe is did it
change
when um in
in in the process of interacting
with artificial beings
such as ai sufficiently intelligent
things
so let's say that in the era of
the interaction and the design of
intelligence so at some point we
discovered that intelligence could be
uh could be designed could be created
and we started thinking look
oh so we can we can reproduce what we
believed
like only god could could do right or
whatever other
theory you want to have um
i believe at that point what we had
was this the realization that
the idea of us being very material
things in the world and intelligence
being something more tangible that's
what it
changed so while interacting or being
able to
design intelligent things uh
while doing so we have had this sudden
argument for materialism that's that's
one of the things that it gave us so
um in the debate about whether
reasons um determine
auction i think certain explanatory
theories such as look reasons are just
descriptions the one that i the one that
i mentioned just descriptions of
physical events
i think ai and technology helped us
in in understanding better
the nature of of cognition
so i'm not sure this answers i don't
i don't think it does
what i was asking anyways but it's uh
it's okay thank you very much
yeah it's truly it's very very complex
and
questions so that's yeah it's gonna
always be hard to cover all the points
right but yeah i think you did a great
job and
we have a next question from sylvia
uh syria um can i read it or would you
like to ask
okay the video oh no there it is it
works
now i was wondering if there is and i
don't know either
how to properly um uh ask this question
either
but um i was wondering if when we talk
about
the connection between reasons and
actions if there isn't first some sort
of synthesis
and the reasons because
[Music]
there can be different actors the the
producers of the technology have
certain reasons that might translate
into for example a different connection
with the actions and a different
meaning or value ascribed to what's
meaningful in the meaningful control or
even what's control
so i was wondering and then the public
authorities might have another one
individuals might have another one so i
was wondering if if
when we talk about connection between
reasons and actions
if we first also need to discuss that or
if
that affects that somehow absolutely
if i may if that was for you the
question
i believe the one that you pointed out
is
one of the first in time
and in priority challenges of any theory
of meaningful human control and control
and
in the time of autonomous technology
defining understanding a framework of
actors and their reasons oh i think this
is mostly what i've been trying to do
the past years actually so i'm
sympathetic with this point
to identify and isolate
the actors and their motives
reasons intentions
that comes before in a way being able
to translate them and find a way which
is which is what we're talking about
here
find a way to understand them
as causes
or reasons or explanations
for the actions of a technology
so this link is the second step
after we have really well
maybe they'll they'll go hand in hand
but it all starts with identifying the
sources
as you mentioned then you can design
to transform these and do the design and
you find a way to explain how they can
influence
the behavior influence i'm using arnold
uh
you sorry um word because it's correct
i like it very much so that's the
that's that's the idea so yeah of course
it's it's actually
the challenge one of the first
challenges i don't have an answer i
tried i
i uh in the paper in 2019 i designed
this um
with felipe with this this
scale of reasons and agents to sort of
try to deploy some sort of framework
that would that would relate to reasons
and actors
in a certain given case scenario
according or donated according to
the nature the proximal nature of their
intentions this is a little bit
complicated
it's in philosophy of auction but you
can identify different kinds
let's say or degrees of intentions and
reasons and
arguments and even laws and values at
some point i don't want to go there
but just like we've been trying
can i can i ask a little follow-up if
there's time
i was wondering because i'm a yuri
and for us there is often the question
of um
balancing values or inevitably
sometimes we stop at okay we have a
trade-off here
so basically the choice then among these
possibly conflicting values is going to
be
one prevailing on the other and uh
what i was mentioning before was more is
is there because this kind of chops off
part of the reasons
somehow but the reasons are going to
still stay there i believe
what we find often is that the conflict
is because these reasons try to
re-enter the whole thing so that's why i
was wondering like
is there a possibilities are there
mechanisms for a synthesis
more more than than that i mean turbine
discusses that a lot and he's one of my
personal favorites because
for the law he is particularly useful
like instead of just chopping off
treating these things as a trade-off
um are there mechanisms are there um
reasonings that actually are more
well yeah i'm more of a synthesis or
more just including everything
from the philosophical perspective i
don't know i mean yeah
i mean there's plenty uh this is a big
problem in
in ethics um so how to do
to take sort of the the best trade-offs
between values and in your case and
norms
um but this is neutral and it's a big
challenge in itself
it's but it's an it's independent from
what you normatively then choose
after deliberation after having
identified your trade-off what you take
for your model of you know we should
follow we should design for
privacy rather than security i'm saying
something absolutely
um random but but the trade-off between
privacy and security that has to be
elaborated
the prior stage in a different context
just absolutely important
it's fundamental but you just sort of
there are two processes different
processes connected
but and then i wouldn't i'm not an
expert in
sort of value trade-offs there's many at
the udel
yeah for sure you probably are a much uh
greater expert than i am at this uh so
yeah two weeks from now we're gonna have
also uh
exactly for instance that's one of them
please join us in two weeks again we can
get to that conversation amazing
so uh um the next question is from elisa
lisa would you like to say or can i read
it
okay are you you're muted
yeah yes thank you i always do that
thanks julia it's really an open
question i have for you um
i'm i'm curious at how also based on the
conversation we just had and then i see
the next
question from claudia cruz
um about how you see philosophically as
a modern philosopher
the difference between control and what
we might call
stewardship um as an interaction
designer control
brings along questions of who
is in control and and who has the
privilege of controlling their poison
power to act
to respond um
it it also brings up questions of
how can i deal you know with with
emerging behavior that might not
necessarily be bad
um so how do i
avoid stifling also um
innovation so so i just
wonder philosophically how do you see
these two concepts or you know what is a
softer version more responsive version
of control philosophically speaking
let me let me take the last thing that
you said it's very
very sort of stimulating uh you said
something about
um there are emerging behaviors
of a technology is that would that be
correct
of a technology that are not
necessarily bad
but they were unexpected and they are
out
of they might be oh correct me yeah
we're open
out of a particular or
a set of particular controllers
intentions right oh we did not expect
this but but it's it's good but we never
we never intentioned we never
did it as controllers so the the the
idea that we tried to
sort of um incarnate with
meaningful human control is that going
back to to what we talked about before
um control doesn't need to be conceived
or at least this is a proposal you can
conceive
control being exercised by
values themselves there at the extreme
spectrum
of the great big scale of intentions and
reasons
right so these values are not are not
persons they represent
uh an emerging sentiment
which is for instance what you said when
you said oh but it's good
right we never did it
in a way but we did it in another way
because that
responds to the
value of goodness i'm not this is stupid
but
to to make the point it's a good thing
so it responds
to this value right um
so in a way that is a little bit of the
essence of the tracking condition where
it says
it has to covaria with the reasons
right it means going hand in hand which
doesn't mean
that if we if tomorrow what's what i
today think it's good it's still good
right and that is the problem that's one
of the reasons why we're here today
because i can conceive control as being
this uh this
alignment between values and reasons and
the behavior
which is by accident good so i'm in
control
but i'm not entirely satisfied with this
because if tomorrow
my values change technology has
to immediately switch
that's responsiveness the way that that
i desire and that i don't
obtain really while i'm struggling to
get out of extremely fully autonomous
systems so the idea of you know you know
you have these two clocks
they're always so if we change the the
the
the technology changes because it's all
i can foresee possible changes and some
of those changes i cannot i don't want
those changes so maybe i have a range of
10
different choices and any of those might
be
okay within that range that's that's a
design question that i really don't know
how to understand i'm sort of i'm
thinking
along with the better designers um
that's that's what i can say here yeah
okay great thanks thank you
uh just let me just correct myself
before i said in two weeks you're gonna
have people from the pool no
in two weeks gonna have steven umbrella
and in three weeks on the 28th of
october we're gonna have evil vulnerable
okay so uh the next question is gonna be
from claudia
yeah hi um actually lisa was right that
uh my question was very much connected
to this
uh to to her question but i do have a
follow-up
uh in a sense that um when we talk a lot
of times about control and also
here when we talk about control and like
restricting um
kind of the outcomes uh possible
outcomes or what we
uh intend for this machine to do
we talk about this idea of like okay at
the end of the day
i have control right but what if
we have a system that you know kind of
departing from autonomous weapon systems
uh going into the direction of systems
that
somehow can suggest to you where to go
so algorithmic systems are like
recommender systems that can provide you
with
ideas for your course of action so in
that situation of course
nowadays in like the the public
discussion that would be
considered to be in control because you
as an ultimate
let's say commander you're able to say
okay well this is the recommender system
and i have to take the decision to for
example
um well flip the the the you know put
the gun or
whatever you need to do so from
your perspective would you say that
given kind of this framework that you
also presented as
do you think that this is you can also
say that the control there applies
because you're there at the end
like doing the action you're actually
the doer even though that actually may
have been
influenced in this case by an algorithm
that provides you with the decision
what to do and uh especially in like a
very uh kind of
intense situation you might not even
have maybe time or resources to
to double check that so so how do you
maybe you know how would you deal with
that kind of situation and control
understanding
yeah i think we have to to some extent
accept that our decisions
as much as our um
opinions are as much as our design
[Music]
ideas or our the way we do the things
they're always influenced by
by something else so i i could say well
in a way think about
your uh decision to
go to a certain
university was it your decision
or was it a decision that was also
partially
determined or influenced by a number of
of contextual cues other persons
so and do you feel that you're
in control of your own decisions
because of that reason it's a question
actually
um i mean of course i do agree that of
course all of our actions are influenced
but in a sense if you especially talk
about algorithmic systems or systems
that you know assist in military
decision making
then you have two very two like two
situations in which first the military
decision making of course has a much
bigger impact than my choice to go to
university
and then you also have a situation in
which a military system or the
the system that i have at hand might be
influenced by design of an external
party so my decisions will not be
influenced by the organization or the
thinking within the organization
uh but externally by other forces so
that's kind of the the tough okay
absolutely i understand can you see this
my screen still
am i still sharing it says your screen
sharing
yeah yes we can so okay can you see this
light here
so um actually maybe i can play this
yeah so that's so to to
to consider the importance of those
normative aspects
of values like well decisions uh
that have high stakes um should be taken
by humans humans should be
in charge uh humans should be ultimately
responsible that is what the tracing
condition does
after so it's not just about the
metaphysical condition of control
the tracking condition that we are in
control
but we need a certain kind of the has to
have certain a certain nature
and what you say is absolutely i
share it i i'm i completely agree with
what you say
and those values that you mentioned
in theory at least sort of the attempt
is that the tracing condition
should set up the
the further normative conditions for a
decision for instance a military
decision where there's high stakes
could be meaningfully
genuinely truly human in the sense that
you can attribute responsibility and the
sense of
so that is a normative set of
of of um requirements rather than the
metaphysical
requirement in the in the link the
connection
so i agree with you and and
that's why we have two
two aspects of control taking care and
you're talking about this
uh aspect on the right side of the
screen
you you can put more conditions here
this is one thing yeah all right thank
you very much
great so i think we we already ran out
of time since we had like some problems
between all this transition from gta to
zoom let's be a little bit more linear
so whoever has to
go another community thank you very much
for joining us
and let's go for one last question here
from arcadi
and then that's it okay i carry
yeah thanks uh so i i've noticed that we
have a couple of more questions here in
the chat so julio if you don't want them
to disappear into nothing
you might want to uh check them
uh before we can send you all you mean
in the chat
yeah i'm sorry yeah yeah yeah so yeah
it's just assuming that my question will
be lost
so yeah i have a very practical question
right so i've been thinking for the past
year or so
how do we actually cover
the behavior of the autonomous of
autonomous systems with the human
reasons and exactly
connecting reasons and then the actions
and the actions we can observe and the
reasons we cannot and then
yeah it's interesting that you mentioned
that reasons are bad causes
uh for action so i'm not sure i follow
that argument but
i have a very practical question right
so assume that uh
we can observe these actions and the
observations are perfect and assume that
we also have some kind of
an understanding maybe not perfect
understanding but some kind of
a causal model if you want of reasons
uh causing actions and then
what we can boil the whole problem down
down to
is basically causal inference so we have
observations and we have
a model which might not be perfect but
we might want to infer what are the
actual reasons behind these actions
given the observations and the model and
then this is
something that we can relatively easily
formulate mathematically and
whether it's valid at all i'm not sure
but what's your take on this
that um i think the problem i mean
there's many problems but
the and challenges not problems
challenges because i like it
um but the main one that i see here
is it stays at the beginning the first
thing that you said oh
let's let's assume that reasons
are things that are good enough to be
translated into causes right um
boiled down somehow transformed
trimmed compressed reduced that's the
term that we use often
reduced to physical causes um
the risk i am at first there's just
several reasons why
they're inherently non-reducible there's
many philosophers um
assuming take this as in a way for
granted i'm not sure i'm not sure
because of as i mentioned there are
descriptions in a high-level language
and the language isn't the same
and when you translate it you lose stuff
that you cannot regain
okay when you translate the language of
psychology
into the language of neural
events let's say i say neural events you
can take physics
chemistry you can go as down as you want
um but there is a loss
of content right and the other concern
still in principle is that it might be
also in
again in principle in invisible
to make sense to make justice
of what we call values
so this is very overarching emerging
entities are they do we actually have
a way to to
to find a counterpart in in a language
that you can then model
mathematically as you said now once you
have that
then you might be over the bigger
problem
but that this is and this is what
philosophy has been doing without
success
much success success enough to help us
at least
um so far this is really like how do i
reduce
consciousness to physical events
oh you you can there's many theories um
that i can accept maybe we can do it
we still talk about mental entities in a
certain way in a certain language
rather than another language because of
reasons
there's reasons to do so oh you can ask
other philosophers
maybe patricia churchland she she might
disagree with me i believe that she
does uh and so on
but yeah i i i own
i see this as being a major problem like
if you're you're assuming
something that is the actual problem
if you assume that's solved i'm happy
thanks a lot julio i think uh yeah we
had some
other uh questions and so i have saved
the answer
i will oh i can send you i can i can
save the backlog i will send you
everything so thanks everyone
for joining thank you for the very
interesting talk julio
thanks so much for inviting me yeah okay
bye
thank you bye-bye