Understanding values – from the perspective of personal support technology (Myrthe Tielman)

all right so welcome everyone to today's
high-tech Agora meeting I assume
everyone could hear me well right this
is our last Agora meeting for before the
summer break so the next time we will do
the meetings we might as well do it in
person
alrighty although we don't know so that
would be in September we will keep
everyone updated through the mailing
list and it is our great pleasure to
have mr. Hillman as an invited speaker
for automating before the summer break
so Murthy is an assistant professor of
the Interactive Intelligence Group in
the Faculty of Electrical Engineering
mathematics computer science Chuck has a
background in cognitive artificial
intelligence and human-computer
interaction
a research interests lie on the
intersection between human and
technology especially on how we can
employ computers to provide assistance
and support for him for various human
problems so this includes social systems
effective technology smart dialogues
knowledge systems and behavioral change
so without further ado Arthur the floor
is yours thank you very much
so I'll start sharing my screen so you
can all see my presentation and you
should be able to see that now
so what I wanted to talk about today is
values and specifically understanding
values from the perspective of personal
support technology I haven't prepared a
talk which very much necessarily tells
you what I've been doing but it tells
you more what I've been thinking with a
fair number of discussion points
throughout so at those points that
definitely I would like to invite
everyone to think along with with the
discussions and the questions that that
I also haven't put on the screen
um feel free to interrupt me in between
if you have any questions or comments or
see anything that you would like some to
see some some clarification about I have
the the window with you all in view so
if you turn on your camera I will
probably see it I should I should also
see the chat but also feel free to just
unmute and talk so like I said this talk
is about values but specifically in the
context of personal support technology
so I wanted to start a little bit by
setting the stage and setting the stage
of that context so the type of
technology that I'm very interested in
is the type that helps people make
decisions and do tasks throughout their
dating life and with daily life I mainly
mean that it's not necessarily for
instance a work setting or a
professional setting but it's it's more
of a private setting where you as a
person are interacting with the
technology so not so much you as a
professional that you as a person that
is very broad so I put some examples
here on on the slide so for instance a
navigation app an app that helps you
choose her out for it for example but
I've also collaborated with people who
work on the vacation app specifically
for people with a visual impairment so
then it's very much more about actually
helping them navigate and knowing where
to turn or where to stop but also for
instance a personal scheduling assistant
to help you keep track of meetings your
calendar if things conflict what do I do
behavior change support systems so
that's the category of systems where
you're really actively trying to change
something and the technology is there to
help you make that change but also even
more broadly speaking you can think of
this general picture that we have of a
personal robot that helps you with tasks
orders your food makes you a cup of tea
and maybe work which is over
you and all of this can be for the
general public or for instance for
people with certain cognitive
impairments physical impairments that
would would mean that they could use a
little extra assistance in some aspects
of their daily lives so this is a type
of technology that I'm personally very
interested in and one common denominator
here is that support in this type of
technology is always about decision
making so you can support someone with
making a choice you can support someone
with executing something so for instance
for the navigation example I gave you
can help someone choose a route but you
can also help someone actually get to
that place if that's necessary but in
all cases the technology is making
decisions whether it's for that choice
or what action to support or how to
support direction
decision making happens somewhere there
and this is then what sort of leads into
the topic of personal values so one of
the most common definitions of personal
values is from the paper in swatch and I
see I missed a mind err I think it's
1992 not 192 but in this paper a day
they use a definition which again comes
from other literature but it basically
says the personal values are the
criteria that people use to select and
justify actions and to evaluate people
and events if we look at values in this
way it makes a lot of sense that if we
have this personal technology which is
going to help us make choices and
perform actions and make decisions about
how it's going to do that that is should
do so in some way in line with our
values because values are two things
that we use to justify these actions or
decisions so to give a little example
here if you want to you know choose
whether you take the bike or
take a car you would want that decision
to somehow be in line with your values a
very easy example in areas say well
you've just told me that you value your
health you value the environment so then
maybe in this case the bike is the
better choice it's just the setup of the
situation that we have we have these
values and we have these technology and
of course thinking about values in the
context of technology isn't the new
thing so most notably the field of
design for values has been investigating
this topic for quite awhile and design
for values has a couple of assumptions
or principles it's based on and I took
the liberty to steal this list from the
deltas I revised Institute website which
I'm also a member of so design for
values has the assumption that values
can be expressed and embedded in
products services technologies system
space and it seems that with personal
support technology this is even more
obvious right because it makes decisions
but even in a way technology that
doesn't make decisions has values
embedded in them for instance through
what choices it offers us to make can
you turn off your camera if you value
your privacy for instance that's a
choice that a system can allow you or
not allow you and it doesn't make
decisions for you but it does have that
value somehow embedded in its design a
second assumption that this conscience
and explicit thinking about values is
important and then it's socially and
morally significant to do that and
finally it says that you need to do this
early in the design process where it can
still make a difference now design for
values as a methodology is in in a sense
that is a little bit older it also comes
more from the perspective of a designer
then it necessarily comes from the
perspective of an engineer and there
seems for it to me there seems to be a
subtle difference between designing
technology and building it and also for
personal support technology there is
this question that arose with me is it
enough to design personal support
technology for values or does the
technology itself also need to
understand values and what's the
difference and this is sort of the first
point at which I like to invite everyone
to think about and jump into this
discussion a bit I don't know if anyone
has any starting words deep questions of
course that is a positive question so if
you don't feel like switching on your
video and participating in the chat you
can type it yeah I see that if Katie
would like to speak please hi yeah thank
you this is a very interesting question
well I mean so just kind of
brainstorming here like when I see these
questions always I think to myself so
with the second question can it be
basically part of the first one that as
part of the process of designing
personal support technology for values
in some situations you might depending
on how you think interaction between
technology and humans can satisfy the
needs in that context of the people
which means you want to satisfy or if
you're talking about needs of
environment also not necessarily humans
so do we see that based on the kind of
interaction that we
to have does the different what you need
to understand the values so I guess it
would really like I think it depends on
the context so those studies
understanding the context is the first
thing that comes to mind the extent to
which would be something yeah and is
then the context of this personal
support technology that is making
decisions about what to support you with
and how would that need to be even more
explicit to make to answer these
questions so I don't know that is the
context you know also could you repeat
that again so just to make sure I'm just
as correct yeah so so you talked about
that whether you just can just design or
whether to know you also needs to
understand very much it depends on the
context I think so say we have this
context of this personal support
technology which is there to help people
make choices for actions is which you
then still say it still depends on the
context of that type of technology or is
there something general that you can
already say based on well so and I'm
probably missing here something because
I know enough with you know the kind of
situations that arise I'm here what you
tell us yeah because what I mean is that
under my context I mean like they
couldn't create social domain in which
like the kind of decisions that you saw
the kind of decisions that we're talking
about so what kind of choices and
decisions are we talking about in what
the realm of life and then depending I
think also what will matter the extent
to which we want to rely on the
technology in that case what
expectations do we place on the
technology so kind of how wouldn't how
do we distribute the roles between
humans and technology because I can
imagine the more we ask of the
technology the more important it will be
for the technology to understand our
values yeah I think that's that's indeed
a good point and that if it's if we
don't
to give the technologies that much
freedom or Liberty or independence in
its choice making then it might be
enough to just have it enable us in our
choices so for instance Jesse was being
able to turn your camera on and off
whereas if it's going to make these
decisions completely independently um
that it might be more important yes now
if anyone else has some thoughts and I'm
interested to hear especially from
different perspectives because design
for values especially is not something
that comes from computer science or AI
but it's very much relevant to AI but it
also brings for me this interesting
point of what is design if you have a
system that's going to make independent
decisions which you cannot predict
beforehand
but maybe it does very deep actually
slightly more practical question and
also so if you look at for a minute like
a design engineer when if you you know
personal support technology if you
design you know the second point the
signals understand the users value I
would say like if you really want
personal support technology right then
then option two would be the way to go I
guess but I don't know how you see that
and like the first point it might be
bits you know if you design for because
because it values can change it with
complex and time and etcetera etcetera
right so is that a difference that we're
looking at for instance yeah partly I
think so for me I I also think the
second thing is the way to go the
question of course is how on earth do we
do this and I have some thoughts about
that which I'll share after it is I
think the the baseline of design firm
values is very good but it sort of
assumes that when a product is designed
it then static and it doesn't change and
if you for instance your personal
dollars change your circumstances change
then you need a new product almost and I
think that given the fast pace of
technology and given also how much
technology evolves after its first
iteration and not necessarily always in
ways that we can predict means that we
somehow need to do more when you still
need to design the initial four to allow
for these values but but my intuition is
that that we need something a bit more
there in that understanding the other
aspect to it is that when you design her
value she typically design for the
values of a group and if you can somehow
embed an understanding of values into
technology then you might also be able
to address the values of an individual
and I think that's a powerful thing as
well which is gained by looking more
towards embedding two values not just in
the design choices but in the reasoning
framework of the technology itself that
makes sense yeah interesting so and
maybe this is a little bit of tension
still so do you so are you also gonna
consider how that if you if you can
understand the users value are you also
consider you know how difficult you will
change that users value to to like use
or is it something maybe a little bit so
we can maybe discussed a bit further
indeed about that I don't know if there
is any final thoughts or if I shall go
to the next topic you don't see any
handsome I'll go to the next slide
actually excited yeah
yes so one are we up so I've been
thinking in my research about this
understanding and what is actually
required to do this and the way what
I've been looking at it is is sort of
three main questions at first if what
what does understanding mean and for me
that means that you somehow have some
sort of internal conceptual
representation of these values I don't
necessarily think that this is the only
way to use values there might be
data-driven methods which can help you
actually make more value aligned
decisions but I have some questions
about to what extent you can pull that
understanding that that's maybe another
another question altogether so for me
I'm really looking at this this how can
we conceptualize values in situated
technology can reason with them and use
them so in order to do that it doesn't
need to just have this concept of values
but it also needs to understand how
those relate to those choices and the
decisions that the technology is making
either with or for a user and then of
course if you have that understanding
the point here is is to make more value
aligned positions and to to use these
representations now one thing I think is
important to mention is that in all of
this I'm not necessarily seeking for the
holy grail of how we can formally
represent what a value means for me this
especially this concept of having this
internal conceptual representation it's
not so much about getting to the point
of what a value is but rather it's a
more practical approach saying how can
we make better decisions so even if we
have some sort of internal
representation of values which does not
fully match
how humans look at values or how humans
reason with values if it's still enough
to make us help better decisions done
for me that will already be a good step
so how you define success there anything
I think is important so what do we need
to understand all use in this discourse
a little bit goes back to the thing that
you mentioned about changing values
right you're talking about a person's
values so I think one of the first big
important questions is how do you even
get get this knowledge how do you get to
know what the person's values are as
technology and also with that how do you
change it right if the purpose of this
is to make it more flexible and to make
it more personalized and changeable over
time then you need to be able to change
these things over time and adapt them to
an individual the second part is how do
you represent values this is more of the
formal question of how can we capture
what values are and how they work more
in formal reasoning snow data system
could use them to affirm a
decision-making and that leads to define
a question of how do you then make these
decisions based on the lowest IQ had so
in my work I've been looking at snippets
to to explore how we could do these
three things and I'd like to start with
the middle one actually because I think
when you talk about how you learn values
it's important to also first have an
idea of what you're trying to learn and
what type of representation you're using
for values so when it comes to formally
representing values I think it's good to
have a bit of an overview of what we see
values as so these are some of the
things that for me characterize values
so there
abstract concepts but at the same time
there globally recognizable context
concepts so they're not necessarily
always have the same meaning for
everyone just 0.3 so different people
might have different definitions of what
privacy means and that can change over
time it can differ per person per
culture but there's still this notion of
privacy which is sort of globally
recognizable to a large extent values
can also be categorized in the sense
that there are some values which are
more similar to others and you can group
them in that way that also means that
they can potentially be in conflict with
other values so sometimes there's some
value pairs which are very typically
opposing so privacy and security are one
of the classic examples and then choices
and actions can demote or promote values
and that's how I been looking at how
they relate to two actions how values
relate to actions so translating that
just to check because jitsi is saying
that my microphone might be noisy can
you all still hear me okay yes yes okay
and I'm not hearing anything we're an
eider so let's just just let me know if
I break up I should be okay so for
translating business representations I
think the fact that values are these
recognizable concepts in itself as a
protocol makes them so powerful because
it means that you can communicate with
people about them and you can
communicate about what it means to be
safe or healthy more easily than if you
just say okay you find this important
that you find that important
having that word to explain
i I think can be a powerful thing
so there's relationship between values
that's sort of how you can classify them
hierarchically meaning for me is given
to values not so much inherently as it
is by relating them to choices or
actions so in a way the way I've been
looking at values is that the semantics
of what a what a value is has to do with
what type of actions they promote or
demote and then this abstract concept is
there for communication purposes mostly
so you can talk to people and your users
about it there's also this inherent
thing that we think that some like say
we have health as a choice or as a value
some choices promote our health more
than others some demoted and some more
than others
so there's this scale involved and you
cannot just say there are two things
both from wealth and therefore they're
equal very often they're one thing does
this more than the other and it can be
both both positive and negative right
something can be explicitly good for
your health or its list like that or
just up nothing to do with it so it is
that we get this notion of we have this
scale which represents how much a
certain value represents a certain
choice promotes dessert choice or
denotes a certain choice the other
aspect is that people prioritize values
in different ways some people would say
my health is more important than social
friendships throughout our people that
could be the other way around there's
personal differences there so alongside
with this relationship between a choice
and a value there's also this
relationship between values amongst
themselves in how much assert a certain
person values them just to say it like
that
so we've been looking at how how do you
represent these things and one of the
things which is which we settle on but
also which is which is commonly done is
that you have both this sort of personal
value profile of a person which just
generally represents their values and
the relationships between these values
and have values relating to choices um
so this little picture represents the
first thing and then you immediately run
into problems because you can easily say
that some people value their health more
than social relationships some people
value the environment more than others
but how do you represent this is it
enough to just write down but then you
cannot really give a distance measure
something is either more or less we
don't know much
maybe we can assume that it's all equal
but I don't think that that necessarily
matches up you can also say okay then we
we give a rating all right then we're
more flexible we can we can represent
that you know in the case of this
picture health is one point more
important than their love of
relationships and that's again so many
points more important than personal
growth but can people really express how
important they find their values in this
way and another question we have is is
this even the same in all domains
there's I think this notion that a
person's values are static and your
priorities are static and yet when you
talk about certain different domains so
whether the way you travel to work
versus how you invest your money there's
maybe it is intuition that some people
will say well you know when I travel to
work
um the time efficiency is important and
maybe money about how when I invest and
suddenly the environment is very
important so this is very much still I
think an open question hmm
and I again liked if I never want to
think along I don't know if anyone has
any input or insights on this party yep
interesting so I don't maybe it's not
about input I do have some further
questions on that first complications
there because how do you see either you
like ranking or rating how can we have
this how can we understand we have any
kind of those things I mean for are we
talking about observing behavior or
asking person can you talk a little bit
of these what is it different what do
you think is more interesting for it
specific context so I have a slide about
that but I'll skip ahead because it is
an interesting question so future goal
later Childress's later also well I
think it's it's interesting also that
you bring it up now because the question
is also does the fact of how you can get
this information matter for how you
represent the information or is it the
other way around do we want to have this
conceptual representation which matches
our conceptual ideas about what values
are and how to behave or do we maybe
want to adapt that to what people can
talk about for instance um I think that
in itself is an interesting question so
so what I've been personally looking
into is is very much the dialogue with
the person perspective but and not so
much data-driven yeah
and so really just just having a
conversation with people or could even
be be questionnaires and then maybe
tweak them based on behavior one of the
reasons I'm not looking at behavior
directly is one in the context of a
behavior change support system that
doesn't really work because the whole
point is that they don't want to keep
doing the same thing and it's very
ambiguous as well yeah people can take a
bike and you can say that taking the
bike is good for the environment and
maybe someone always makes choices that
are very good for the environment but
maybe they're doing it for completely
different reasons that they like the
fresh air they live close and they don't
want to spend too much money in
electricity so drawing that explicit
link from behavior to values instead of
the other way around I think is can be
used as input maybe at some point but to
base your whole system based on that I
don't I can't see that clearly yeah yeah
I totally agree with you and it's very
challenging if you look at any kind of
data-driven I don't know any kind of
learning approach it's very hard because
that's it's on the car of these and the
ambiguity problem is everywhere and the
consequences are very hard but just if I
can just I saw that Cariaso and assume
that you're gonna follow up very quick
question given this okay given we can
understand the profile of the person but
you mentioned also that the the present
the meaning that they make of this value
can be different for different
individuals right meaning that they make
often in the specific context as your
example and privacy and everything so
I'm wondering cuz it's very clear for me
sometimes that when we use the values
within within like a within a social
sciences that you can only talk with a
lot of people and understand everything
but for computational season I struggle
is in a
because I understand there's different
conceptualizations of privacy for
example with different views but an
agency's they had do we need to settle
on one in order to make some workable
thing because I mean your preference is
on privacy but what privacy how can you
understand in the video
meaning 8 you know values yes I think
the next slide goes into the little bit
so maybe ok so and then we'll go back
the next slide is discussions - that's
good
ok so I'm really hitting to the flow of
your presentation so later for the end
ok thank you I think our car do you want
something yeah really quick question
first like I'm really fascinated with
the whole narrative so far so I love the
representation of this small human there
as for numbers I mean this is but still
so this reminds me of a character in the
computer game where you basically say
this this vector of ten numbers is what
the person is but that's not a point but
this connects to my question which is
basically it is really I'm really
curious about the way you could actually
capture the values and represent them in
in an artificial system but to me it
connects like 100 percent to the
question what are you going to do with
this representation I am afraid that I'm
running a little bit ahead but it does
seem to me like this is really really
abstract an interesting theoretical
problem but when it becomes tricky is
when you start asking question how are
you going to use these numbers am I just
going to make life-or-death decisions
based on these numbers or yeah yeah
comment click on that
that'd be great yes I think that goes a
little bit into the the
importance of decision-making and
reversibility which is not something I'm
necessarily addressing here yet but I
think that that's indeed important
because there there's always this sort
of measure of like you're not entirely
sure a letter if you just use it this
way it will be correct right because we
lose something if we represent values as
numbers and yes I say I think we gain
the possibility to weigh them against
each other which is maybe even more
important but I will go go into that a
little bit more in the question of how
we use them a bit later on I see two
more hands so feel free to turn on a
camera mic yeah well I'll keep my camera
off because my insurance is sketchy I
think this approach of designing with
values like super interesting and
especially what interests me is when it
comes to value conflicts you mentioned
the conflicts sometimes under different
contexts but what also interests me I
would love to have you thoughts on this
is when it comes to conflicts between
values of the consumer or the person
using our products and then of us as a
creator so for indeed we might have a
very different understanding of what
justice means of fairness right and then
yeah I think it's very easy to see this
walls that go like this little perfect
place where like although he's aligned
with these of our users but and also if
you look at all the political conflicts
in the world right now it becomes
apparent that the understanding of what
fairness and justice is for instance
also like caring for the planet is very
different so we deal with this if our
values as designers or engineers don't
align with our use of areas yeah so I
think so so what's being done in
designer of values is very explicitly
that you know design for your own values
you talk to your users about their
values right
designing for values with your own
values and - it's really beside really
misses the point of that whole thing
I think in a way representing values
explicitly goes even further to actually
help with conflicts because you make
them explicit and whereas in design you
know you talk to your users and you talk
to what their values are what the
conflicts are and you try to find the
best solution having a system that has
these values represented explicitly I
think really helps in making these
conflicts more clear and make the
decisions more clear and I think that
that doesn't solve the problem but I do
think it's the first step is recognizing
the fact that there might be a conflict
and my perspective as well as always
very much been that you want this
technology to adhere to the values of
the user but there is always the
question of how far do you want to go in
that so you one if you have you know a
self-driving car and that person doesn't
value the environment their own safety
and just wants to get there fast and
that do you don't break the speed limit
probably preferably not right so there's
always this trade-off but I do think the
powerful thing of having values
explicitly represented behind those
choices is that you can make that
trade-off clearer and then also tweak
whose values you're going to find more
most important incidents in both choice
and I think that's something that's
harder to do if you're just designing
and even harder to do if you're not
thinking about these things explicitly
at all yeah it's interesting if we then
have the right to challenge the values
of our consumers as well kind of
acknowledging them but then nudging them
or at least arguing to push them in a
certain direction yeah
well that's also what happens now right
and especially with more to healthcare
type of apps they sort of all assume
that you want to be healthy and
there's one goal and they very
mindlessly go towards that or as this
perspective of putting the user central
stage would also mean that that you can
maybe lose that that goal but I think in
the long term as people will want to use
this technology more it will be more
effective but you also yeah you have to
think about what well as a society we
want and whether it is okay for us to
have personal technology which adheres
to our values but which in the
background be sort of running what
society is all once of you and also
tries to fold that message because I
think then you also get it to a bit
Shady terrain of how do you trust that
the decisions are made for you or is it
secretly a decision which sort of feels
okay for you but also slightly precision
of their each other yeah thank you did
you have fun yes I know I think we just
lost him Mac can you hear me
yes sorry that's a big connection
connection I know so I was wondering
given the trickiness of constructing
such a profile
could it be that instead of aiming for a
very accurate representation what if we
aim for a human machine interaction that
lets the user build the representation
in an interactive way so then we let the
user have an authority and autonomy of
building a profile that they identify
with so it's basically like if the
ultimate goal is to have an assistant
can we have can we fulfill that ultimate
goal without the very tricky task of
constructing such a quantitative
representation I think I think we can
and I think that's very much in line
with the point I made earlier off of
what is how do we measure success
do we measure success if we have this
abstract representation that perfectly
captures everyone's values or is it
enough to have something that's useful
for our interactions and decision-making
to make them at least better and I think
the perspective indeed of letting the
user take central stage in that and
having something which maybe not
perfectly represents their values but
which at least they have control over
and they can input can be a good first
step at least and then maybe you you can
have some more data during measures that
say hey there's a lot of people who
would associate this choice with
something demoting this value you say
you find very important is that actually
the case or is something else happening
here and then you can maybe tweak it
yeah I've been becoming more attentive
to - like opening up our design
imagination I kind of build beyond so
kind of bringing in the interaction of
the human and technology and really not
always trying to kind of solve
everything through perfecting the
technology but actually give giving more
giving more authority it's all me and
responsibility to the human that is in
this interaction in a way that's
ultimately together - then we actually
fulfill the kind of the bigger goal that
we want to fulfill in this case yes
no definitely so I'll move on a little
bit and I'll touch upon that exactly
point a little bit later so so a quick
note and given time I'll skip the larger
discussion but this slides goes back
into the how do you give meaning to
values right so the previous discussion
there's the discussion of how - what
values do I prefer over others but
there's also that how do we give meaning
to what these values mean
so something that we've been looking
into is relating them to choices and
that means that automatically you need
this representation of what your
possible choices are and maybe even
their consequences so having this
explicit representation of choices
allows you to then make an explicit
linking with values you can say that
there might be some ground truth in what
values is enjoys promote or demote I
think to some extent there's there's a
baseline for that taking the bike is
better for the environment right but
there's also some values for which it's
harder health and the environments are
fairly clear but what is good for your
value of Independence can be very very
different for a person so I think that
having this individualized meaning to
what for instance independence or even
privacy mean is something that's
important and then of course we we again
get this question of can we attach
numbers of how much is something does a
choice promote or demote values and then
you get exactly the same problem that
you want to say this hurts this value
more than something else and yet
attaching numbers is a very difficult
thing and especially because if you're
taking your perspective of talking to
this from a user perspective right and
that's what we've been doing very much
putting users center stage here so one
study that we did on this was with this
context of visually impaired travelers
the context of an app on your phone
which helps you to to navigate and with
within this context we wanted to know
okay if we have this representation of
choices which in our case will
sort of a formal hierarchy of behavior
so for instance at the top you would
have go to your doctors and then you
would have several different ways of
getting there
and then these different ways of getting
there
for instance by bus or walking would
have different parts so a part of
walking there for instance could be
crossing the street a part of getting on
the bus is you know getting to the bus
stop waiting for the bus recognizing the
right bus number getting in that's all
right so by linking actions that way we
have this representation of what the
choices are you can go in different ways
and you then need to execute certain
things and with their structure we
wanted to attach values and in this case
we skip the numbers for now but we just
said okay can we talk to people about
first of all what these hierarchies of
actions look like for them personally
because everyone is different travel
behavior and then secondly can we talk
with them about what values certain
choices promote or demote so we used a
conversational agent for this to some
extent a visual representation of these
actions would help but of course for
these users that's impossible and the
advantage of working with visually
impaired users is that they're very very
used to speech systems that's how they
interact with all of their technologies
is true spoken and spoken text for
text-to-speech and speaking back so they
are fairly competent I would say more
competent than your average user in
working the conversational agents so
what we wanted to do in this particular
experiment was really to figure out what
would happen if we talk to people about
these complex formal structures with
complicated concepts such as values and
the fact that a behavior can be a way of
doing something or can be a part of
something
we can have these nice formal
representations of user models but if we
then want to talk to people about it
indeed what happens so this is
preliminary work in the sense that we're
not trying to perfect the system here
but we're trying to figure out what
happens and what to watch for and what
we found in both the questionnaires that
people did and the interviews that we
did with people after they have this
this very structured conversation with
the conversational agents about these
structures and these concepts is that at
some point you can have the situation in
which the user is not actually talking
about the same thing as the agent
anymore so that we called misalignment
and of course if that goes on to the
long ban the misalignment will basically
happen in the user model that you end up
with basically the user model that you
have will not match what the user thinks
they told you or what is right what
their own representation is that's of
course something you want to avoid you
want that to be as similar as possible
and what we found that was interesting
specifically with the idea that you have
these complex structures and concepts is
that there's different layers of
misunderstanding that can happen in the
conversation so the very basic is
understanding the structure we had these
hierarchical structures with values
linked to them and understanding how
concepts were related if there was a
mismatch in communication about that or
people didn't grasp that concepts were
related in that way and it led to a lot
of confusion about what these concepts
actually meant but also even though just
knowing what to say to the
conversational agent and if you
understand a question answer something
different and then the system doesn't
get that you misunderstood that gets
into the system and the errors that you
see in this picture are basically house
misunderstandings could lead to others
so misunderstandings in what concepts
meant what it means to promote or demote
a value could then lead to not knowing
how to answer it and to assume
misunderstandings again so I think this
was useful because it shows that if you
want to talk to people about these
formal conceptual representations you
need to be very sure that they
understand the structure of what you're
talking about
and that they understand the concepts of
what you're talking about
and then of course there's the whole
practical problems with having a
conversation with an agent which always
occur but there there's a deeper problem
in where people don't know how to answer
not just because there is problem with
the the speech-to-text
but because they don't conceptually
understand the user model that you are
trying to construct with them so I think
that was our biggest lesson here and I
think that ties into a couple of things
that were mentioned before and that if
you're on the one hand you want these
formal representations which match
conceptually with what you're trying to
model and on the other hand you need a
representation which is can be followed
by people so so that's kind of a tricky
thing right you don't know there's any
comments questions
so and this was them one of one of the
discussion point I had which adidas can
you learn values also automatically or
and to what extent can people talk about
their values explicitly how much do they
really understand what underlies their
actions we are fairly used to thinking
about these things but a lot of people
have never heard of the concept of
values in the first place
right and if you ask them what value
does this promote without giving more
explanation they'll see say I do this
because it saves me time or I do this
because I'll get there quicker and of
course there's an underlying value that
is quite clear that that is about the
value of time and time efficiency
maybe but getting there quick is not a
value in itself so that that's a tricky
thing I think so I final the final point
was using values which we touched upon
and all round up because I see it's
almost time and then again we get this
question of can we use these numbers
right we want some way to represent how
much something is valued and yet numbers
feel like an oversimplification I
haven't got I have I don't have a better
answer than numbers I think most people
who are working with values in such a
way are using numbers but it's good to
keep in mind that this is we're making
some assumptions here about what values
are and how they work which are not
trivial and to what extent does it
really is this really different from
just using a utility because that's
basically what we're doing right and I
think the fact that you can can talk to
people about it on a conceptual level is
what separates it from a very simple
utility but
in a way we're just doing the same thing
and can we do away with numbers in
Scituate that we don't constantly need
people say oh this I find more important
this is no no I want to do this because
the large part of the point of this
exercise is to avoid that I think this
was my last slide so maybe in some given
time some final questions remarks I
think we have more questions than
answers still but thank you previous
question you mentioned discussed and all
so I'm wondering what's your opinion on
the more data-driven perspective like if
you go into the donor had the chance to
look the beneficial beneficiary I had a
new book from Stuart Russell yeah he
argues the point that I mean of course
the the the AI should be always
uncertain what your preferences are they
should learn from your behavior things
very clear from then its behavior is the
ultimate source of information for them
that's the thing that can be
questionable but they also say ok we're
talking about value alignment but in the
end it means preferences and in the end
of it are just gonna values are what you
consider important and those preferences
are values I think I always have a
little bit of problem because I see this
as a different concept that we talked
about this before but my question goes
in ways you think that it could be
everything we discuss it could also be
towards preferences regular preferences
or and in the other way do you think we
can ever get to some kind of efficient
upgrades that are we'll be able to
understand so well what do you think
yes I think for me Andy there's there's
this intuitive difference between the
value and preference or as a preference
is more temporary somehow and the value
is a more long-lasting stable thing I
can definitely see situations in which
you use values in a way that you could
just as well have use preferences and I
would say that especially now we don't
have the full conceptual richness that
values have inner representations yet so
they're more like resemble the richness
for me personally comes in to the fact
that you're not communicating with
people about their preferences but
you're communicating with them about
their values so it's almost that the
communication part with the user is more
important there and how you talk to
people who are types of terminology you
use in talking to people about what they
want that that's more important the
other aspect is especially with like
behavior change and the what what people
prefer now is not always in line with
what they tell you long term
preferences are the way I see them more
short term I would prefer to have all
the ice cream but I value my health the
way then one a system that's going to
bind you all the ice cream or do I want
a system that maybe goes against my
concrete immediate preferences sometimes
in order to support the more long-term
values so I think those are two
important things I think learning values
from only behavior it's much more
difficult than learning preferences from
just behavior also because we don't
always act in line with our values it's
how we judge your actions that's
important when it comes to values notice
to do just what we do if that were the
case that we'd never want to change our
behavior right if we always act along
with our values and perfectly satisfied
with that I do think that behavior and
more today to prove a measure
can be very powerful in fine-tuning
systems like these and in making sure
that they're not as intensive to use as
something that constantly asks you to
get your input on what you want the
systems do I think there's there's a lot
to say about having a lot of interaction
with your user about what you want and
to make sure that it understands your
internal representation of them fo and
vice versa but you don't want to tell
your system everyday every time
something changes and that's what I
think we're the day the Durman approach
can be powerful to fill in the gaps and
to to connect the dots but also maybe to
see ok this person just said that they
want to do this but they're always
acting completely differently and when I
make a suggestion they ignore me is that
because I'm fundamentally wrong do I
need to change my strategy for
convincing them but at least trigger
something like ok maybe I need to do
something here so I for me that's the
parts where I would see the data-driven
approach being very helpful yeah thank
you a really yeah I agree with
everything you said and I would say I
think one more thing else towards
explain ability I think it's so much
easier to explain in terms of values and
norms that inventor explains in terms of
like my preference on attribute 47 is
0.3 like what what are you talking about
I have no yeah like I said for me values
it's it's it's not even just about the
concept it's also about the fact that it
because it's a concept you can
communicate to people yeah and if we
look at your values also we can
incorporate so much discussion on social
science and ethics that connects to that
so that's ok thank you very much other
questions or comments
yeah if any I yeah later thanks a lot
for a really interesting talk also by
the way really cool illustrations
I got I got a tablet to draw and I at
home because I never have a whiteboard a
really cool thanks yeah and I think very
thought-provoking and so so for me like
that connects to what I was raising in
the previous question so the question
that your third question on the previous
slide you know kind of the balance
between the new miracles
so the yeah using the numbers for values
and and needing the Constitution and for
me like like I'm thinking like somewhere
in between is kind of what what I'm
curious about because because for me
something that I really realized that we
we too often try to quantify things that
in my opinion we really should not try
to quantify because as you said it can
feel often and it is an
oversimplification I'm curious about
your opinion about like when you think
about personal assistance to what extent
would you like them to see empower
responsible decision-making on the part
of the human but where the human really
realizes a at the end of the day the
personal assistance is just an advice at
the end of the day I'm the one who is
taking a decision and it's my
responsibility and I'm curious about
your opinion of others I understand that
of course there are circumstances when
we talk about people who may have
certain disabilities for example where
we might actually want personal
assistance who have more authority to do
something ultimately that might benefit
the well-being of those people but yeah
I'm curious about yourselves
yeah now I think it's a very interesting
question
I think that and there that's also why I
like and I took a girl project because
there's there's this trade-off between
the power of intelligence systems is
that they can do stuff on their own
without input from us that's also
immediately to the downside of it and
finding this
balance between letting them make
autonomous decisions and execute them so
that we can spend time on other things
versus making sure that we still have
control over what's going on it's
something that I don't necessarily have
an answer to it is one of the reasons
that I think the explainable transparent
part of AI is very important and it
indeed matters a lot what the decision
is on so if the decision is on you know
this person needs to go through the
doctor am I going to say them to them to
turn left so they can take a bus today
or right so they can't walk there is a
completely different the implications of
making the wrong choice are different
than in some other situations right so I
think that that is an important thing
what are the implications of choosing
wrong and maybe also having something
that and maybe values could also come
into that but knowing you know what
could go wrong if you have two choices
to make where both of them you know not
choosing something can really demote
your values but you can only choose one
so for instance if you need to choose
between two important meetings and then
the fact that not doing something not
doing one of them if you need to not do
one of them can really demote your
values can be an indication in itself
thank you maybe need to let the person
make a decision right so I think values
can be an interesting aspects there so
and and the other part of what you're
saying is really interesting because
I've actually been I've been talking to
some people about this concept of
explain why for people with disabilities
and there you get the question of how
transparent do you need to be in your
decision making right which is a very
difficult question I think
to answer but one that's important
because you need to know whether you can
make decisions for these people or not
pisum in certain situations yep
thank you Thanks any final words
questions I think most people need this
to leave it for Cara - yeah yeah I think
we can wrap up now so you're - thank you
very much it was very nice and so yep
thanks for having me I'm pretty
interesting uh interesting discussions
yeah make sure - yes yes
back in September we're gonna take care
everyone


8 Signs It's The Future: Thought-to-Text, Nvidia Text-to-Video, Character AI, and P(Doom) @Ted

I want to know if you agree that each of
these eight developments would have
shocked you not just six months ago but
even six weeks ago these all came in the
last few days and range from text to
video thought to text GPT models
predicting stock moves and AI
Annihilation discussed at Ted but we
start with nvidia's new text to video
model rather than show the paper I'm
just gonna let different examples play
on screen from the paper one of the
breakthroughs here is in temporal
consistency essentially the series of
images that are used to form the video
are more aligned with each other so the
sequence plays more smoothly with fewer
sudden glitches or changes the generated
videos by the way have a resolution of
1280 by 2048 pixels rendered at 24
frames per second and there is a
powerful line from the appendix of the
paper that was released with the samples
the authors say that they expect
enhanced versions of this model to reach
even higher quality potentially being
able to generate videos that appear to
be deceptively real they go on to say
this has important ethical and safety
implications this future might not be
far away as I'm going to show you in a
moment with the progression that has
happened in text image in one year just
before I move on you may have wondered
when this is going to become a product
well they kind of admit in the appendix
that they can't yet make it commercially
viable because it's not ethically
sourced it was largely trained on
copyrighted internet data and yesterday
blockade Labs showcased the add to this
feature where as you can see you can do
Doodles and turn them into images that
go in this 3D World we are swiftly
moving from two Dimensions to three
dimensions and as a bonus this is zip
Nerf a 3D neural rendering Tech released
this week I'm not even counting this as
a third major development I'm lumping it
in with the blockade labs this video
shows what happens when a series of 2
dimensional photographs are merged into
a 3D drone like video and probably a
real estate agent's dream just imagine a
cherished moment in time being
crystallized into a permanent immersive
experience and soon to be honest many
people may not have to imagine with the
Apple reality Pro possibly debuting as
early as June and costing around three
thousand dollars apparently according to
Bloomberg it might be available to buy
in the Autumn and have things like a
dial where you can move between virtual
and augmented reality coming back to
what has already occurred do you
remember when a mid-journey image won
the Colorado State Fair digital art
competition well now the same thing has
happened to photography the
quote-unquote photo on the right was
generated by Dali 2 and it won the 2023
Sony World photography award now the
artist behind it Boris L dagson did
refuse the award I want to show you a
few images that show how far mid-journey
in particular has come over the last
year because many people believe that
mid-journey version 5 is actually
Superior to Dali 2 which won the award
take a look at the progress of the
different mid-journey versions and
remember that version one was released
in February of last year it was almost
exactly one year's difference between V1
and V5 here is another example of the
progress and at this rate we will have
mid-journey version 50 within about 10
years what will that version be like
before I move on to the fourth
development I'm going to show you two of
the craziest images that I could find
from mid Journey version 5. I would say
I can still tell which images are AI
generated around 90 of the time but my
prediction would be that by the end of
the year it will be 90 of the time that
I can't tell if you thought text to
image was getting crazy what about
thought to image or even thought to text
here is a fascinating extra tracked from
the AI dilemma they took human beings
they stuck them into an fmri machine and
they showed them images and they taught
the AI I want you to translate from the
readings of the fmri so how blood is
moving around in your brain to the image
can we reconstruct the image then you
know the AI then only looks at the brain
does not get to see the original image
and it's asked to reconstruct what it
sees so when you dream your visual
cortex sort of runs in Reverse so this
means certainly in the next couple of
years we'll be able to start decoding
dreams okay so it can like see
reconstruct what you're seeing but can
it reconstruct your say what you're
thinking your inner monologue they had
people watch these videos and would try
to reconstruct their inner monologue so
here's the video is this woman getting
hit in the back getting knocked forward
what did the AI reconstruct
I see a girl that looks just like me get
hit on the back and then she's knocked
off the fifth development concerns
something rather more mundane which is
making money now many of you may know
that AI is already well on the way to
conquering poker I used to play a lot of
Poker myself and dare I say I was pretty
darn good at it but even though poker
involves predicting human behavior AI is
starting to master it which brings me
nicely to the development I actually
wanted to talk about which was
forecasting the stock market according
to this really quite interesting paper
accurately forecasting stock market
returns is an emerging capacity of
complex models I'm gonna let the author
of the paper tell you exactly what
prompt they use so the exact question
that we ask is yeah financial advisor
here's a headline is this headline going
to be good or bad for the company in the
short term once you have enough
headlines you basically just invest in
the companies with good headlines and
not invest in the companies with bad
headlines and here is the table
summarizing the results A1 was a
positive headline a negative one was a
negative headline and a zero was a
neutral headline and when gpt3 analyzed
those headlines you can see the
correlation with the average of the next
day's return that's positively
correlated for good headlines according
to Gypsy 3 and negatively correlated for
bad headlines and you can see that
earlier models really couldn't do this
and as many of you may well be thinking
this is gpt3 this is not gpd4 I bet that
as we speak there are thousands of
Traders using the gpt4 API to predict
the next day's stock movements I think
the results will get even more
interesting as the context window
expands and Gypsy 4 or GT5 can analyze
entire articles press conferences Etc
the next development is that fairly
quietly without attracting many
headlines role-playing chat Bots are
beginning to go mainstream character.ai
founded by one of the original author
offers of the Transformer paper recently
crossed a hundred million visitors that
is starting to resemble the growth
trajectory of the original chat GPT gpt4
is still smarter and better at doing
these role plays but the interface of
this site is easy to use and of course
it's free this is not sponsored in any
way but it was pretty fun playing this
text-based adventure game I especially
enjoyed going on complete tangents to
what the adventure was supposed to be
about and I think confusing the AI the
seventh way that we can tell that the
world has fundamentally changed is that
Google doesn't seem all conquering
anymore as it has done for my entire
adult lifetime first I heard in the New
York Times that Samsung was considering
replacing Google with Bing as the
default search engine on its devices I
am not surprised that that shocked
Google employees then yesterday this
article came out in Bloomberg it says
that many Google employees beg their
leadership to not release Bard saying
that it was a path a logical liar and
cringe-worthy I've done videos on that
myself but even when Google employees
tested it out asking it how to land a
plane or do scuba diving the answers it
gave would likely result in serious
injury or death in February one employee
said the following in an internal
message group Bard is worse than useless
please do not launch the concern that
many have is that now Google will ditch
safety in order to catch up as the
article reports these staffers who are
responsible for the safety and ethical
implications of new products have been
told to not get in the way or try to
kill any of the generative AI Tools in
development which brings me to the final
reason that we know we're in the future
the risks of AI Annihilation are
beginning to be taken seriously it's not
just loan voices anymore like Elie Isa
yudkowski who a couple of days ago got a
standing ovation at Ted he believes the
world is firmly on track for AI takeover
it's also other senior figures who
believe our probability of Doom from AI
is non-zero here are some selective
probabilities but I want to focus on
Paul Cristiano who gives a risk of Doom
between 10 and 20 he previously ran the
alignment team at open Ai and now leads
the alignment Research Center and you
may remember them from the gpt4
technical report they were the guys that
open AI trusted to run the model
evaluation of gpt4 testing whether it
could autonomously replicate and gather
resources which they concluded may
become possible with sufficiently
Advanced AI systems but the conclusion
is that the current model is probably
not capable of doing so these are quite
senior figures and insiders giving a
non-trivial risk of AI Annihilation I
think that deserves a lot more public
conversation than it's currently getting
and on that Sam Altman seems to agree
and the bad case and I think this is
like important to say is like lights out
for all of us uh yeah I think it's like
impossible able to overstate the
importance of AI safety and Alignment
work I would like to see much much more
happening so the future is here and I
think the media needs to catch up


Meaningful human control over automated driving systems  (F. Santoni de Sio) - 1st AiTech Symposium

there you go so hi there how are you
today
so my name is Philip Poisson Toni the
CEO I'm a colleague of your room from
the novanet dissection philosophy of tu
Delft and I will tell you something
about this project I've been running for
two years in particular on the
philosophical part of it the definition
of meaningful human control and in this
work of course I've started working on
that i'm with yoren and then I've been
working mainly with Giulia makatsch who
is sitting somewhere in the room and it
is partially responsible for better for
worse or what you would be hearing now
by working with your own I've learned
something very important about
responsible innovation about the
sensitive design this is one of my
favorite quote from your own and other
colleagues in depth when I moved to
death that was something really struck
me but what you know as a philosopher
we'll talk innovations that define
innovation if you talk to engineers
innovation sounds like do new stuff new
functionalities new gadgets something
that works better but in your own and
colleagues case say hey this is a very
limited conception of innovation like
real innovation is when you can break a
trade-off between values you want to
achieve something you want to see
something else
current technology doesn't you allow to
what she borrowed these things you make
a innovative design that allows you to
achieve both of the things you want to
achieve right this is real innovation
according to a certain view of
responsible innovation in in order to do
that as I would not repeat that you need
a love of a different approach to
technology you need more in
disciplinarity more of a design
perspective also in philosophy you need
to take social science and empirical
studies very seriously you need to
reflect on technical in institutional
design at the same time so a lot of
challenges that a itec project as I
understand it will try to take on and
also you need to have word at our
faculty TPM we call comprehensive
engineering so I think our faculty is a
good example of a first attempt of
realizing an institutional place where
this can happen because under the same
roof you have complex system engineers
you have people studying the human side
of systems multi actor systems and your
people like us studying the human side
of it the value side of it economic
value philosophical value
ethical
says sayfudine security and so forth now
indeed a mere Philemon control idea came
from philosophical point of view from
this idea of breaking a trade off like
if you talk to people in different areas
about autonomous technology you will
hear this sort of a the atomic answer
say sorry if you want to go for autonomy
then and for efficient innovation
apology but incidents will happen
will happen and you know this thing of
human responsibility you man
accountability this kind of overrated
like why do we want responsibility let's
just go the way for efficiency
innovation and on the other hand you had
very conservative people you know
technophobic people who say no no we
don't want any of that because we want
to stick to all possible safety and
human accountability so the holiday
meaningful you my controller is again
hey why should we choose why can't we
try to redesign the whole process of
design and regulation of autonomous
technology in such a way that we can
achieve both a maximal level of safe
safety and accountability and all the
efficiency in innovation we want to
achieve so that's a basically push of
the project and again this is the
project we are in it's an Embraer
project we are partners and we have
engineering traffic engineer engineering
social psychologists behavioral
psychology and philosophy working
together so again this is sort of a
attempt to realize this idea of
responsible innovation of course this is
a background of the of the specific case
study namely automated driving systems
but and this is the set up of the
project so basically we are moving away
from the robot dilemma to some
definition meaningful human control we
have these three disciplines we have
three use cases and we are trying to
answer these three related questions and
in particular Julie and I are busy with
a philosophical conceptualization of
meaningful human control but as you
don't say this is not our term the term
was coined in the political debate on
autonomous weapon systems and in
particular the NGO article 36 came up
with this notion and everybody was super
happy about that
so that was a very interesting political
phenomenon when a certain term magically
started attracting
consensus around it and then the problem
was as a philosophers that while you
reading the definition of these terms
everybody's going their own way and I've
been to some of this discussion as
information and at some point we were
like look we need to find a
philosophical definition if you want
this concept to work if we're like no
don't do that
otherwise we will stop agreeing right
that was there is this interesting thing
I've talked to people involving this
process that they don't want to define
the term because the fuzziness of the
term is what allows for disagreement but
as philosophers and designers we do need
and we do want this clarity and this
possibility of translating the concept
into design requirements so maybe in the
I'm not an expert of the diplomacy and
politics about autonomous weapon systems
per se so maybe it's a good idea to keep
it vague there but it is not here in
Delft and in other technical
universities so the story of meaningful
human control is at some point people
were very concerned about the
possibility of having fully autonomous
weapon systems for the reasons that here
you mention and this is a definition
among many others or what an autonomous
weapon system is which is already quite
a controversial thing as you can imagine
and basically there are two main
concerns with autonomous weapon systems
where unpredictability what if as you
don't say at some point a target is
identified as relevant and it was not it
was just some you know AI messing up or
sometimes it does you know the other
hand what if that accident happens
people are killed civilians are killed
and there's no way to reconstruct the
chain of accountability which in a
military and political domain is super
important as you can imagine so there
was this idea of meaningful human
control and this was the general
definition of it so humans not computers
and their arguments should remain
ultimately responsible and disease they
of course the difficult part for
potentially lethal operations a critical
function that in a dimension in in the
beginning so this is in a nutshell the
result of many years of reflections that
Ian and I had in this paper 2018 so at
some point you and I thought look we
have this experience of working of free
will and moral responsibility for many
years there's a lot of theories out
there and some of these theories
specifically focus on the conception of
control at the level of dividual human
being in order to be responsible for
your action you need to be in control of
your action right and so we try to use a
specific theory in that fit I will not
turn you into that at this time of the
morning would not be a good idea but we
took some of the criteria from that
specifically official a visa and we
translated it into criteria and Spanish
and translated that into criteria that
could work for the control the
meaningful human control which grants
responsibility over autonomous systems
and reserve the two conditions we came
out with tracking and tracing which mind
you this is disclaimer they do not
necessarily mean what you think they
mean in engineering or your discipline
so it's a specific meaning of that and
by tracking we mean in the system
conceive of us human operators operated
device infrastructure so the socio
technical system should be able to could
be designed in such a way to be able to
cover its behavior with the relevant
reasons of the relevant human agents in
the network of the systems this is a
tracking condition I will get back to
that and the trusting condition is
supposed to cope with the accountability
problem we want at the same time that in
any of this socio technical system by
design there is at least one human agent
which at the same time can appreciate
the technical capabilities of the system
so as some sort of a reasonable
understanding expectations towards the
behavior of the system while at the same
time also appreciating here on moral
responsibility for that so we want to
prevent the responsibility gap where on
the one hand you have engineers who
understand everything about the system
but they they come they don't consider
themselves responsible because they've
 the responsibility to the users
while at the same time you have the
users who do think they know that they
are responsible for that but at the same
time they can't appreciate the
technology enough as to really be
responsible as in satisfying the
conditions of capacity control on the
system itself and indeed you don't need
to go to very futuristic autonomous
weapon systems to
that with very low levels of autonomy
you can really have already big problems
of human meaningful human control and
the Tesla accidents this is a stupid
axiom as you know there have been way
more tragic accidents with fatalities
involving out test autopilot just as one
example and there the problem was
clearly DS that was a big response moral
responsibility gap from a legal point of
view that was settled the driver was
responsible idiot you should have had
your hands on a on the wheel case closed
but from more point of view this is
disturbing because this driver hadn't
received any training was not fully
aware of course you signed a lot of
terms or condition as we all do without
this technical assistance but it's
really dubious way that he had some deep
moral responsibility in the sense of
knowledge control etc that you don't
mention so basically we started realize
that those in the driving systems there
is a big problem of definition of human
control and this is a standard
definition of autonomy and what is a bit
concerning about this a set of
definitions about autonomy is that it
seems to suggest that the more you are
on this side the more control you have
and the more you are on that side the
less control you have which is sort of a
heuristics that in that's not
necessarily work because Tesla is here
too and this seems to suggest that the
driver is in total control just because
he has his ends he's supposed to us his
hands on the wheel but as we see this is
not the case so that could be a mismatch
between our definition of control from a
technical sense as he not said and our
definition of meaningful human control
the kind of control the grounds moral
responsibilities and so indeed this is
sort of a traditional controllers we
non-engineers understand the engineering
notion of control right is as far as
there is a responsiveness of the system
to the action or behavior to a
designated agent human or not there is
control but this is not meaningful human
control why because this applies very
well to old-style dumbo systems but does
not apply to everyday systems this is a
metaphor a variation of the horse
metaphor of Flemish in which you say
okay is it clear who is supposed to do
what in order to achieve what from a
train but here who's in control of this
specific horse here is the organ itself
because his smart intelligent autonomous
is the specific driver of the horse is
the audience around is the person in the
audience was training the horse and so
on and so forth is the interaction of
all of these elements how do we define
control there this is the challenge of
meaningful human control and our answer
is in a nutshell that our tentative
answer which is a very broad framework
which would be implement implemented in
different contexts with different tools
etc the general idea is by using this
tracking interesting condition diseases
and a meaningful human control to the
standard it responds not to the action
but to the reasons so a more abstract
level not the behavior but the reasons
behind the behavior the values the norms
the intentions of who some designated
human agents which may be the user the
controllers could be the designers could
be so we're also broadening the scope of
the potential agents who can could be
deemed as in meaningful human control of
a specific system and at the same time
there is at least this is a threatening
condition at least one human agent who
can be legitimately called to ask for
the wrong behavior of the system so we
should design vit this example in such a
way that by design we can reliably in
the most early show that the relevant
element of the system are responding by
design to the relevant reasons values of
the relevant agents so it's it's a lot
of work right that's why we're here and
at the same time that there is at least
one person there be that that guy in the
pub in the audience because they the guy
here the trainer of the horse the
organiser all day of the fair whoever it
is who was entrusted with responsibility
not all in the legal sense which is the
current solution now let's just decide
that you are responsible you pay and we
will discuss it in the past week this is
sort of a legal scapegoating right we
don't want just to decide a prior that
someone will pay this wouldn't be fair
so we want to entrust people with real
responsibility as in the capacity in the
awareness to realize these
responsibility conditions by design so
in a nutshell this is they just grows in
our talk with some specific implications
of it
this means that we need to really have a
broader conception of the different
technical and human components in a
system moving to a broader understanding
what the system is not just a device by
the institutional system around it the
network of people are only debt cetera
identify the social players in the
values that we may want or not want all
in advance by design designed to realize
this interactiveness an interaction
between human and robot and David will
say more about it I guess in training
humans also lay people to realize this
control condition identified this is
more a psychological part of it
identifying the necessary human
capacities for and relevant knowledge of
a given control task you name it what as
creating effective mechanisms of public
accountability so this is just I'm going
back to how they did the story it
started this morning within up this is
very multidisciplinary enterprise but we
hope that with this notion be specific
interpretation of the notion of
meaningful human control we may have
contributed to have some steps forward
in this complex task thank you
thank you Filippo Santoni the CEO and
neroon from there over and we have a few
minutes for burning questions who would
like to give reaction or ask a question
to one of the two former speakers ego
can you please speak into the mic great
presentations it's just working yeah is
there an example where meaningful humour
control has already been applied to a
large degree so many of the aspects
you're mentioning are already on the
table and have been discussed you mean
as applied to some autonomous takes some
autonomous system that's already out in
the society that's a very good question
we as philosophers we tend to focus on
things that do not work so it would be
nice to have a positive example of that
let me think about it
yeah I cannot come up with one specific
example I can give you the sort of the
idea I mean for instance if you have
something working in a very controlled
environment takes out one automated
driving systems for instance I guess if
you say if you take there's this project
in the Netherlands the we part project
and indeed it's a very gradual attempts
to get to sultanim allah knows driving
but step by step and so for instance we
have this shuttle who is unmanned but
there is a remote controller sitting in
a control room and this is where for
instance you have this idea of combining
a professional expertise as opposed to a
layperson on board sitting without
pressure in a control room operating a
system which is moving in a controlled
environment because the environment
itself has been designed to not present
challenges that the vehicle cannot
address so I guess this is in principle
a good idea or they or the system design
a minimum control of course if you look
at my graph at our graph this is very
much on the safe side and possibly not
so much on the efficiency innovation
side because cities say hey you know if
you talk to enthusiasts about
self-driving cars say yeah but we don't
need that we already have trains if you
want to have self-driving cars on trucks
there
we better stick to trains so I do
understand of course there's a push to
but the idea is that you should go step
by step
once the the thing works in this control
environment you can say this is the
challenge of the project you can
slightly start introducing variables and
testing them and also the testing part
is very important he didn't mention that
the testing part is very important too
so that he'll be a sort of an example to
start from a control environment by
design and they're removing obstacles
well that's kind of where we are right
we have robots in all the controlled
environments and we want them to be in
society right now so lapis is where we
need to really think about this indeed
right right behind you one last question
you're gonna and Philippa will be around
for part of the day so but we can take
that question please I would like to ask
a question about okay it's nice to have
a meaningful human control but what if
you cannot expect a meaningful human
control I'm thinking here about for
example nursing homes where people with
dementia are supposed to be empowered
and supported with autonomous systems
semi autonomous systems what then how
does your framework work in that kind of
setting so you're assuming that it is in
this second meaningful control would not
be possible
well I know the person from the my
cognitive impairments
cannot know fulfil the criteria not that
person but a part of the theories
exactly that you may shift a meaningful
human control to other persons like you
may have controllers somewhere sitting
and you may have the design of the house
be obviously the device we are such as
to not allow for things that could be
you know detrimental to that privacy
well-being or the person so my first
reaction we may be meaningful in my
control is possible we need to study
that okay
maybe of course if we that's a very
great example because if you focus only
on the obvious controller maybe you
cannot achieve it but maybe there's some
other way and the other answer is
unfortunately if at some point Sri turns
out that technologies is not allowed for
meaningfully much control in a critical
setting we may decide not to go for it
okay thank you thank you very much both
of the speakers let's thank them again
[Applause]