Nicola Croce: ODD, data labeling and the problem of representing knowledge for AVs

uh discuss while i'm talking because i
just put together some
eight or nine slides but it's just
something for me to
guide my my talk and discussion but
again i'm very much
open to an interactive session also
because all these topics are
very much open-ended and there's no
right quest the right answer yet
and i don't know for a long it's gonna
be like this so we'll definitely
good to also gather your feedback and
thoughts on the problem so
let's start so basically uh we'll go
through uh data labeling
and uh from the from the point of view
of knowledge representation so
uh we can see data labeling as mostly uh
knowledge representation problem which
is that if we just stick to
what it is in in practice because um
so basically this is the agenda we go to
data label for first principles some
open issues with that
with that point of view and then odb
also has another
knowledge representation problem um
two premises i i also
try to urge you to disentangle a little
bit
the view of data labeling from the
specific
machine learning model algorithm
technical implementation
i know that the two are very highly
related but
it's i think good to just try to
abstract away from that
to see the thing for what it really is
and then
just the goal is to provide a general
concept of framework of what labeling is
uh mostly for concept learning which is
the branch
of learning machine learning and
learning that deals with learning
basically uh belongings to categories
of the world and not regression those
things were just saying
what is what so which which object
belongs to a certain category
so uh basically what we see um
when we label data every day and also
and my company deeply is that every
labeling task every leveling job every
every customer comes in with very
different specifications so
for some people they're just interested
in
five six objects like pedestrians um
cars trucks uh buses
bicycles and motorcycles for example for
others they're interested in a
20 plus categories with them very rich
attributes that are ranging from
state attributes to position attributes
to
visual appearance attributes there's a
wide
variety in what customers and people
want to be labeled according to their
how they think about their system and
and things like that
but ultimately the problem of labeling
is basically very similar to the problem
of language
which is uh being described a bunch of
years ago i believe in 1923 or something
like that
but to gentlemen's with this semiotic
triangle of language
which basically says that whatever
whenever we
we give meaning to things and and build
a language out of it we
do this exercise of taking a symbol such
as a string of characters like the word
car and
and which stands for basically we want
to substitute
a real thing or a mental model with this
symbol
so we have the symbol card that stands
for a real car in the real world
which is not exactly my i don't know for
example this
opel that i have in front of my window
now in front of my
view outside of the window but it's just
we want to
stand for the concept of that of that
open so we want to abstract it away
and mode and use it through a thought of
reference
that leaves just in our minds of course
and that
abstracts away the characteristics of
the properties of that
referent so that that real object and
stands for it through a symbol so this
is basically
exactly the exercise that we are trying
to do when labeling
right so we're just saying having this
list of things of categories that we
are interested in and we have data
and we need to figure out a way to
basically say
to teach to cars or to machines or to
autonomous systems
what is what the problem is that
this view is um so if we look at this
from these lenses
it becomes really a knowledge
representation problem right which is
the problem of
how we represent human knowledge in a
way that is
fungible for machines and understandable
kind of a machine and they can also
maybe reason upon so
the promise then becomes more complex
because the
reference so this card is not living
anymore
into the real physical world as we
humans perceive it
but instead it leaves into inform what
they call information containers or
or files right that organize the
information in certain specific ways
for example here we have you know an
example of a video stream
which is just basically a collection of
individual frames right
without without ordering time so every
frame is an image
and every image is just a collection of
three mattresses
one for red one for green and one for
brew
which every matrix matrix has a value
left so every entry in the matrix has a
value usually from zero to 255 to just
specify the intensity of that pixel
right so
we really need to understand that these
machines
have only these mediums or these
limited information that comes from
three matrices
to understand the concept that you're
trying to tell them
hey learn this and
so for example a very trivial let's say
or
the very widely used task
for machine learning is is called object
detection
and it's basically the task of
localizing
specific objects into a scene through a
bounding box
in order for the argument to learn that
as i mentioned we just try to strip away
from the technical details and i don't
want to even
call it too much objective action but
what we do from a labeling perspective
just
go through all these frames and mark
and specify a bounding box around for
example cars
this is our way of pointing a finger
and saying hey algorithm like
this is the concept of a car that you
need to learn
so we are basically using what i call a
labeling construct
so a tool to summarize in our view
the concept that we want the machine to
learn and
and point that finger as a for example
as a dad
that says you know points a figure to a
currency and teaches you okay
this this thing is a car we do the same
thing by just
doing this bounding box which is
basically cropping the image around the
the concept that we want
the machine to to learn right and this
basically is a tiny extension to the
triangle that we saw before because we
we use
one one step more which is
going through this labeling construct so
dissecting the
container of information which is the
frame in this case
and and find a representation
of the object that is worthwhile for us
that
in our mind summarizes well enough the
concept for the machine to learn
of course a bounding box is is not that
full of knowledge because it's just
basically a smaller
picture right so the of course this is
also the problem
the big issue of explainability and and
whatsoever in a
machine in neural networks but this is
what this is exactly
all that mineral network has to
understand and to learn what the car is
of course this labeling construct is is
very important i believe because this is
what basically the
the way that we represent human
knowledge
for the machine so this is the way we
represent these concepts
and that's why for example this is not
the unique
uh way that we have to do so
uh if we stick to the digital reference
so this
digital thing of images so you know it
could be jp
jpeg files png files or whatever it uses
this rgb let's say matrice matrix
matrices
we have multiple ways of refi of of
saying
uh and of representing human knowledge
the easiest one is for example
which is usually refer attached to the
data the computer vision task of image
classification
is very simple it's basically we just
attach a string
so for example here the string dog to
the whole container of information so
to the whole picture we don't use any
labeling construct to summarize
a concept we just say to the machine hey
this picture is a dog
go figure it out the problem is that
this picture does not contain only the
dog right it contains
a lot of other stuff with which are
context noise and things like that
and this is the job of the neural net
right the job of the neural net has
is to just figure it figure out itself
what makes a dog a dog
and and that's basically the simplest
task from a human perspective because we
don't represent knowledge
basically at all we don't do any sort of
feature
engineering whatsoever we just say hey
go figure it out yourself
this is a little bit problematic
sometimes okay we know that
theoretically in machine learning
theoretically
if you have infinite data you're done
like
the neural network will solve this and
we'll will just understand
uh to be able to uh detect dogs in
whatever image you
uh you submit to them but the problem is
that practically we're always bounded by
uh by a data quantity somehow and
and we saw there was a paper that uh
also an experiment that a friend of mine
working in the field in machine learning
decided me
actually yesterday there were on one
hand imagenet is performing which is a
famous model of neural networks which is
performing way better than humans
on this task of image classification
on the other hand in practice there's
going to be been a lot of cases where
for example
uh classifier were trained only in
images of cows
uh which has we which had the background
of
scotland uh you know fields and then
if you if you so if you feed the network
with an image of a cow in a silly for
example you will never be able to
recognize that
because it of course the model the
thought of reference
of the of the mental of the neural
network which is basically what we are
trying to
come up with so we are basically using
the construct to summarize a
concept but then we have not we have no
control over this thing for a neural
network the neural network will figure
that out
itself and we don't know what this thing
is right
so uh this is the problem of
explainability but this is
also something that i think we can um
damper and dampen a little bit if we
think more deeply
about how we represent knowledge for the
neural network to learn from in
principle so i talked a little bit
of this simple thing where the labeling
construct
so this aspect goes basically to zero
and we just attached a string so symbol
to the whole image
then we talked about bounding boxes
there's another way
that we have as humans to represent
knowledge in images which is
uh usually it's attached to the task of
semantic segmentation but this is not a
computer
vision task but this is just label this
is just how we represent this concept
and which is basically in this case as a
list of pixels so for example this car
is just a list of
of all the pixels that compose the car
concept
and this is for the trees buildings
pedestrian etc so this is a little bit
more refined
for example that than this
because we we have a little bit more
granularity in the information of the
localization about the localization of
the entity
and what it really is made of but for
example
this also from how we label from a
human knowledge representation is a
little bit problematic
because what we are saying here to the
algorithm is not
this is a car it's just that all these
blue pixels
are made of the cardis material kind of
thing
in this in this picture in this task the
model is not able to distinguish
between instances it's just being is
just able to distinguish between sort of
let's say materials of stuff so this
piece of the image is made of
uh source of pedestrian material but
this thing is
exactly treated at the same way of this
the algorithm doesn't know that these
are two
different uh basically entities so
it is reason in a very different way
than human the humans right
and that's why the whole point of what
i'm trying to convey here is that
we probably need to think more deeper
deeply about how we do labeling
first because whatever we do here is
basically
a human representation a human knowledge
representation of the concept that we
want the machine to learn from
and uh these things might not be
rich enough so the
world in autonomous systems i think is
now dividing itself into two main
directions
one is going towards end-to-end deep
learning
so these things are not even
done or are done just in a minor form or
for specific subtasks
but the whole goal is just to have an
end to learn pipeline that says
this is the raw censored input and this
is the exact
for example path that you need to follow
and then you as a deep learning model
need to figure out
everything else so you this is also
partially like implemented through
reinforcement learning or a little bit
of
in imitation learning which is basically
you just label data by
driving right but there's no
there's no human representation
whatsoever
there's no even human understandable
layer whatsoever between input
and output everything is in the hands
100 percent of the narrow of the deep
learning system
basically this is great on one hand
because it
it really we're seeing some
breakthroughs and it really could be
you know a good way forward there are a
bunch of startups
uh carrying out this model but of course
this is a huge
issues in terms of explainability uh
safety and and liability because we
don't we really don't know what
what these systems are thinking about we
really don't are not aware about
understanding what's going on inside of
their mind while on the other hand
the other path that i see in the
industry is
going through more so-called
neurosymbolic approaches neurosymbolic
ai
where we think a lot more about how we
represent knowledge
for machines and how we encode we label
things
and how we modularize the technology so
that we
can have always uh keep an eye on the
uh basically on the pieces of what is
the input and output from the blocks but
also we can
open the black box and see inside of it
and this is
also the path that a lot of
big players big oems uh like uh
you know the the traditional car
manufacturers are converging towards a
little bit i believe
and and that's why we need to think more
about representation so for example
what i can envision being happening in
in the future in this field
this is my personal view of things but
is that
we we started seeing something in the re
in the research going uh towards the
directions that
will build basically semantically rich
description of the scene with uh we
which may
will probably make use of mediums like
knowledge graphs
or or either even eterogeneous graphs
that gives a very complete description
of this thing
in terms very close to natural language
ideally we should say like this guy is
on this lane which is next to
this other lane is running towards
this other guy so it's basically a very
semantically rich
representation of the scene which gives
us more power
on understanding and also on giving the
networks and the models and
our technology uh the idea of which are
the
concept that we care about as humans
right
so why i told you all these things
basically because
there's also a moral aspect to this
attached to this i believe because every
day that i
i in my job i see new labeling
specifications coming in
from from industry players
and sometimes i find like uh the
distinction between
child or pedestrian sometimes i just
treated
as pedestrian in in any case right like
i just want to label podesta i don't
care about
even if a child or not so
do we need to care about it i mean is
ethically
something that needs to be regulated
because at the starting point
i envision being the standard for label
for example now we are building at this
standard for label which is
non-normative non-prescriptive
whatsoever just uh you know the first
step toward the direction but tomorrow
i can envision an authorities coming up
and say hey
you you really need to show me that
you're not only be able to
detect pedestrians for example but you
you need to be able to
detect whether the pedestrian is a child
or no
uh of or even or if the pedestrian is
working on crashes or no because these
things can have an
impact on the season somehow but
also uh the you know these
entities can behave differently and can
have uh produce some
risks associated with them that are
differently according to
what they belong to so this is basically
what's going on with labeling and why we
can
i can envision this being very important
decision from an ethical and moral point
of view
and this is basically
uh a list of open issues that you see
with these things that
of course every knowledge representation
so every
access that we do with labeling is a
surrogate
as we saw first we are limited by the
organization container
of the information so that if we have
rgb if we have a png file we just have a
png file with just the
three matrices if we want to tell more
to the model we need to instill more
human knowledge in that one
because every knowledge representation
is is not a completely accurate
representation of the object right
always we are always losing something
the other problem is that of course
this is a trivial problem i mean this is
something that everybody knows there are
infinitely many educators in open world
so that it's very hard to put things
into categories into buckets
because as you label data and tons and
tons of data you will see that
even if you work a lot and you put a lot
of effort in engineering the taxonomy
the ontology behind it
and the the categories that you're using
you will always
find a weird edge case in your data that
you
you're still wandering and scratching
your head and say oh okay what's this
thing how do i call this thing is it
is it this category or is it probably
doing this action or this other one so
you know we need to find ways to and
find tradeoffs
to deal with this problem and we have
some uh
ideas there also in the standardization
i can tell you about if
if you are curious but this is something
that we cannot
avoid like this this is the problem is
there somehow
so then again what does make a good uh
ignore
uh chaos and knowledge business a good
and fair one this is the problem they
just mentioned you about should they
distinguish between child and
pedestrians or for example or is
is not something that i should mandate
and then of course
the other the final problem i want to
throw at you is that
all these things have strong algorithmic
and technical traders and constraints so
the the models how they work is
very very dependent of the
representation so for example the
the neural network architecture of an
object detector so of a
thing that is trained and developed to
to draw bounding boxes around things is
very different
from this thing so these two models are
very different
like the these they work in a very
different manner
which is tied to the the starting
representation that they have to
in in as input so there's a problem
because
if we change drastically this the input
representation
then we need to to be to do
to spend a lot of effort in finding new
architectures
finding new models that can deal with
that representation so that's why i
think it's a
it's a hard problem and i i'm seeing
some research and some papers that are
that want to go towards a richer
description of the scene
but they just find hacks to
basically fit their representation to
the model instead of
trying to come up with new models of
course because it's very hard and
restores the money
right and then that's the hard thing to
do coming up with new
cutting-edge technologies and and and
architecture that can deal with these
things
but i think that this is probably
something that will open up a lot of
breakthroughs if we do so in the near
future
finally the odd is nothing that us
yeah before we move to the odd can i ask
you a question
about these points i mean
um it's more of a common
probably but like what i was wondering
from a non-technical perspective is
that uh one of the issues also is that
uh
this knowledge representation problem
like also the way we
uh label uh data that you were saying
like is a child as a person with
crutches
um isn't it also inner to the fact that
um we don't have also a
diverse pool of
people doing this kind of work i mean
how do you see this in your
experience and the kind of assets you
get
to work with yeah that's that's a good
comment actually because we see
so for example there are two things to
be said first is that
so for some cultures you know they are
interested in different concepts or you
can see that you can tell that
uh their taxonomies their specifications
are made
through a specific point of view uh in
some ways
for some it's very it's very of course
evident when you're talking about
geographical locations right like
if we get data from india you'll have a
lot of categories of waves
vehicles that you will never see in
europe uh
that's just an objective thing of course
but
also these um i i do think that these
uh are are definitely also guided by by
some cultural aspects because for
example
i see so personally i see some when i
see data from
our taxonomies from uh you know specific
oems in europe
uh in a specific country in europe they
they tend to be very precise
in or try to
to put a list of very rich attributes
sometimes not in the best
organization possible uh but they try to
to really
provide a rich rich representation of it
while in other countries
they're very they're much more
streamlined
of course this is a technical cultural
aspect but
the the major aspect here is is due to
the
i think to the experiences of the teams
and how they
they go about building neural networks
because at the end of the day
if you're very granular in what you want
to represent you need to have
enough data or of that concept to be
able to infer it so
usually the trade-off is
stripped a little bit the taxonomy so
that you have enough
instances of those things that you want
to represent in your
in your that you want to infer and learn
in your data because for example
if i want to come up with a category of
uh
horse instead of just animal
well i need i need a lot of forces in my
data to be able to recognize the next
one
right so this is basically a little bit
of the trade up there
yeah we also have a question from
luciano
okay yeah thanks so uh yeah
a secretary of follow-up a little bit of
what lucha asked
so there is of course the problem
there's a lot of different people can
label this differently but also with
time these definitions might change
as i said there are infinitely many edge
cases in the open world
and also with the etch it is that those
labels definitions they can
change over time so what we consider in
a car
today is not the same as you consider a
car 20 years ago or something like this
so yeah so i'm just wondering how do you
see this because it's hard to get a
you can get a good and fair knowledge
representation that you consider good
and fair have enough
data but that might drift apart with
time
yeah that's uh that's a very big problem
um
for example uh you sometimes it happens
that customers
label data in a specific way with a
specific taxonomy and then they figure
out
one year later two years later that uh i
wanted to add this thing or i wanted
these things to be different for this
this reason
and they have to re literally throw away
all the work
and and re-label everything which is a
lot of
it's a big cost so data labeling is is
not cheap
because it it involves a lot of human
labor and
and this is a problem that we see in the
industry so what we are trying to do
to partially solve that is not this
problem is not
going away easily but partially solving
it by
coming up also with the standard called
opening scientology
which tries to be a very
comprehensive and detailed
ontology of the domain of road traffic
so that we are trying to put definition
of what a car is
what is built for what what are his
parts like
wheels doors whatever where where a
truck starts and a car ends and how are
all these things related so a real
ontology basically a knowledge graph
and and openly so the standard of
labeling will make use of that ontology
to label concepts in a way in such a way
that
at least partially good or all the new
for example
data set can be mapped to each other's
thanks to the ontology and uh
if i am a manufacturer that i'm using
one version of the ontology one subset
of it let's say i want to
extend it or modify it i can use
this big monster to just map my concept
around
but of course this is not the fantasy
like this doesn't solve
all the issues but it's at least
something like a first step towards
also tackling a little bit this this
issue
okay yeah thanks thanks
uh we can move forwards um
i think yeah okay yeah it's like um
very close to the end because uh for the
odd uh what i wanted to say is basically
this console operation design domain
that
is is nothing new it's basically
stating the conditions under which my
system
is designed to function properly and
these conditions are
you know general like um environment
condition
um type of uh
road that i want that my system is
allowed or to function or is designed to
function properly
and uh for or weather condition
etc etc and also some modes of
working such as speed limit for example
or
type of traffic agents that my my
system can deal with so if you think
about it this is nothing new right
also my microwave as an odd
it says or my iphone don't don't use
uh into the inside of the water for
example
or below three meters in the water or
or under a strong sun that burns it
circuits this is the ldd right this is
basically exactly the same thing the
problem is that now here
we have to deal with odds design around
things that
behaves in the side autonomously and it
has
to deal with an open world context so
this make the problem
pretty hard to to treat um
but this is basically collapses to the
same problem of not representing
knowledge because we need to come up
with a proper odd taxonomy we need to
come up with a proper which is basically
a list of
okay how how we enumerate things so
that we need to list in our odd so my
system isn't is allowed
to function we when rain is
up to a certain threshold or when fog is
uh below a certain threshold so
it's really a big problem because you
need to categorize everything you need
to figure out which are the right
categories
etc etc so that
there are for example there are two main
approaches
for odd definition one is
bottom up and one is top down the top
down one is kind of problematic because
exactly you need to start from the
bottom and say okay what's my led is
based on
is made of scenery elements so
rolls junction roundabouts so you have
to enumerate that
is based on weather conditions
which are the weather condition need to
enumerate that be able to measure those
and etc etc and then it's very hard you
will figure out if you take a map of the
wall you figure out okay
what's the highway here or what what
really when
when when this is a country road or when
when this starts to be um you know uh
i don't remember the name but there are
specific names that uh a lot of
standards have come up with for four
types of roads
but you you'll never be 100 sure
that your street is of that
category for example so the trade-off
here is to manage
balance the top-down and definition with
the bottom-up
the bottom-up is basically okay we know
that odd has to deal with with the world
somehow with the
with the geofence area so we just say
i want to design my system for the
san francisco area so i need to
map san francisco very completely so
this is concreteness
substance test my system the whole
area of san francisco and then
try to abstract a little bit away some
of the categories that make up
san francisco dd such as um
highways off-ramps off-ramps
on-ramps off-ramps and and those kind of
things
uh but this is still um you know you
need to find that
they trade off and there's still an open
debate around
where to put the boundary how to build a
decent enough odd taxonomy uh
for generalizing the odd and and
bringing them
everywhere around the world and then how
to properly
instantiate it into a specific area and
go very concrete in saying
uh what's my odd because
this is also what you really end up
doing is testing and designing a system
for a very specific
geolocation let's say that then you can
abstract away go to another location
instantiate it there
and re-test partially your
system to make to account for the
things that you didn't manage to
abstract and that are very
typical of that specific reason and
that's very concrete
so there's this let's say concreteness
abstract nice concreteness process that
you need to go through
when you when you expand or you scale
your
automation capabilities to new location
there's something pro that's probably
something that is going to
happen at least because um i see that in
the standardization
world where we are trying to standardize
also open odds to a format that
describes ldd we are
really you know bumping our head on the
wall
to to try to figure out how to make a
trader for this problem and
probably this is going to be the
direction so find a good trade-off
between
very concrete odd top top sorry bottom
up
design and a very abstract ability
design top-down
so i will i mean there's a lot also that
more than can be said about odds and
what's in it and what's outside but i
will
open up the floor if you have any
questions or would you like to
talk about specific aspects of it yeah
is there any question from the audience
for now
otherwise i have a question related to
this so i was looking oh yeah there is a
question
um let me see
please yeah yeah i'm i'm just gonna jump
in quickly because i have to go to
another meeting
i just wanted to thank you first nico
for the for sharing
uh your thoughts and i think you're
you're pointing to a lot of really
important um challenges and addressing
the
kind of normative dimensions of like
what it means to build a proper
knowledge grab when we think about
especially about safety that's my kind
of my concern
um i just want you to know i'll reach
out to you
uh because i think there's some
interesting connections with some of the
work that we're doing
is title hard choices where we're
thinking about what are these
kind of trade-offs that are built in how
do you
um kind of combine
approaches that are more semantically
rich like you're doing
that are more data driven and then how
do you bring in like
different kind of value perspectives
um and that's what we are like kind of
kind of combining like
philosophical um kind of
inputs with like um the more technical
development practice
yeah um so i i have to run but i just
wanted to
to say that like thanks for sharing this
and i think there's a lot a lot to
follow up on so i think that's great
thanks for looking forward it
thanks for the comment um
is there other questions or i had the
curiosity
oh i've got any of gummy
from the beach yeah yeah santa monica
nice
i wish hi nico yeah
thanks very much uh for the talk i was
uh so this relates
more to the to the beginning where you
talked about labeling
um i i was curious about your thoughts
so
i i think much of the many of these kind
of issues that you talked about were the
deep learning models for example they
are not capable of
demonstrating the kind of understanding
that we humans have
a lot of it comes from fundamentally
different
way that kind of these models operate
and we in the following sense that these
models at the end of the day
uh make classifications and predictions
based on statistical correlations
so it's really correlations of input to
output
but we humans operate in a much more
sophisticated way which is uh more like
causal
reasoning which implies additional
levels of sophistication like
what's a cause and effect and and
control factual thinking like
what if i had done this like in a
different way what would be the outcome
but these models are simply not capable
of this kind of thing
so they might see indeed like you said
if they only saw examples of
cows with green grass they might
learn that a cow is green like
the green pixels rather than what is
actually the animal
of the cow um
what what are your thoughts about uh you
know
kind of trying to combine these
perspectives because also
trying to adopt more of the causal
modelling
kind of way of thinking it also has an
important role in making these models
more explainable
opening them up for scrutiny it invites
people to make
assumptions more explicit about what do
we put
into our models yeah now this is a great
remark
so i think well the whole field of
causal
learning also by asian networks and
everything is i
is i think is progressing while we are
seeing a lot of breakthroughs there but
i i so personally i i think that we'll
we really need to convert those towards
these neural symbolic approaches where
we
combine the perceptual capabilities of
statistical models as you mentioned with
some
more richer new knowledge representation
in a cohesive way
and why i say this because it's because
even if we're
when we talk about simply statistical
models
in the exact same time that we are
labeling something
we are representing human knowledge like
we are
trying to get the model to think as
ourselves
even though the model is designed to
just and
its nature is dna just to find
statistical correlation as you mentioned
if i label a car i'm doing i'm biasing
it i'm saying
you need to learn this human concept
somehow so
uh that's why also i think that whenever
we
you know we hear uh deep learning
uh gurus or you know the founding
fathers of the
field like ben juan and everything and
all these people
that has done great achieving deep
learning
sometimes i still feel that we
there's not really this acknowledgement
of the fact that
yes everything that is supervised has at
least a little bit of human knowledge in
there
so that we cannot avoid that that piece
of of trying to let things
learn our concepts to to to work with
them
so if that's the case might as we might
as well just
uh be go towards uh a better way of
doing it
and and adding human knowledge in a
more rich way you know in a more
complete way
and figure out how to plug them into
these perceptual or statistical things
at least that's my personal opinion and
i'm very ignorant in the field of course
because i'm not the osho banjo for
whatsoever uh but this is how i view
things if
i just sit back and look at what we are
doing
um i i still think that we we
it's really hard to strip away human
knowledge for whatever we do because we
just are
wired to think in a way and we want
machines to
try to think in the same way at least
partially well and i think
just like speaking for myself for my own
experience i think like for many years
i i kind of uh
you know i think we often go with the
with the
uh um how to say this like
we give the technology uh qualities that
it doesn't have
like in the way we think about it like
for example that we might expect that it
has this kind of capability to represent
things in
our human ways but actually it doesn't
and and i i
find that like as somebody who comes
originally from the technical field
i think looking back for years i like i
would think no no but the neural network
will be able to
like magically you know learn uh and
only i have to say only in more recent
years i really
started started understanding these
limitations like right like machine
learning at the end of the day
correlations this is not causation and i
think this is something that also an
education
of uh engineers is something that
can be clarified much more and it will
help it will help us
think uh i think more creatively also
about the waveform yeah
no that's a great point also it should
be clarified because
and the problem is also marketing right
like
when we hear when we read titles like
gpt3 has solved
human intelligence just because it's a
very very good language model
it doesn't have a clue of what he's
doing it's just a fact like
it's just predicting the next vector of
of values so
and also if we there was some some
debunking uh papers about it i mean
it's very good it's very useful it's
super powerful also for
the bad things if possible
but that's not intelligence right
that's not intelligence is i don't know
what it is but it's not that thing
uh it's just another model that it's
extremely good with a ton of data like
it is 13 billion parameters or something
to predict the next string
of word after what he has seen
and it has you know he has seen a lot of
stuff so
probably is gonna do well but then you
see that for example
if you type some addition or or
operation then you you go and check
where it trained and you see some blog
posts where
people were just doing this operation
and just learned to copy that
it didn't learn about to reason about
things so that's
that's why this is definitely true
we also need to be very clear in
communicating what these things are
really doing and what they are not doing
because it also when you see
some startup pitches from startups in ai
that peace to investors
but sometimes you really feel like
don't say this thing because this is not
what you're doing this is just
you're basically i'm a good model to
statistically correlate this thing in
this thing
it's not solving the intelligence
problem this is not
how we are selling the thing right and
and this of course is uh is a great
issue but if
so if you guys are interested i know if
you have a look at it but i had a look
back in november or december it was the
ai debate with the banjo
ma gary marcus and the others and
it starts to it seems that a lot of
people
that used to be very very super deep
learning
uh let's say fans
or you know traditionally the the funny
fact of it
i started to realize probably we need
something more we need some more
we need to get go back to some more
symbolic approaches and figure out how
to
merge the two words and and those kind
of of things
because there are some very strong
limitations that we cannot
uh still tackle and i'm very lucky to be
working in autonomous vehicles domain
because i think that
this is the field where you really bump
into these problems and
hopefully someone will will
you know solve it somehow the i think
that this is the
the testbed for these things because you
need to be very very smart
so it looks like and a lot of companies
usually that build just ada systems so
driver
assistance systems like coma ai comment
rather they they just
you know their voice is really driving
is hard to ask but you know
it's not that hard we don't have
spinning liners on the head don't have
this thing you just solve it with a
camera and you're done
yes you're building a level two system
so you're you always fall back to the
human
in certain times if you talk about full
autonomy
whatever it means driving is a very very
very very very hard task to solve
we have another question from luciano
and
yeah before we run out of time
yeah okay second question so first let's
let me just say this
discussion here you just had here it's
really interesting i
really like this where it was going so
i'm gonna bring this in a little bit of
a
different direction here my question is
a bit different but just i just want to
say it's really nice and it's really
important for us to understand if you
wanted to
just delegate all the tasks and all the
responsibility to machines
and where are we as humans so where
can we still be responsible for the
others for any of the behavior if you
cannot really understand
do so really nice my question is um
when you you were talking about odd the
examples you gave they are
a lot like um looking for to the outside
we'll talk about the roads if it's a
highway or not
if it's about the the pedestrians that
are there
so there's there was of course a little
bit on the representation so
but i'm just wondering should this odd
so the the
ontology you're working on are you also
looking
inside the car so looking who is driving
what are the
emotional states of who is driving so
should these kind of
looking inside be part of the od how do
you see this
yeah thanks for this comment that's my
point of view i'm trying to
influence as much as i can the people
also the the community i'm working with
in the standards because
it looks like traditionally or the
the majority of the industry industries
is leaning towards odds and
everything outside nothing inside but
it's weird because i don't think it's
the right
way to of approaching it and uh
why true thing first is that internal
states are very important
today you have um light bulbs
like you know saying hey you need to
check your
lights or something tomorrow you don't
have this you have a system need to
figure it out so
there are some internal failure states
that could be part of an odd
now the discussion is yeah but those are
just failure stage we just
call them failures failure modes and we
and we
strip away from odd the problem is then
okay but these will probably affect
how you how you
behave and also it can they can put
conditions on odds so there's this
notion of conditionality
if you have the battery is i don't know
over 50 degrees because of overheating
something
then you probably need to restrain your
odd and
and your strain your velocity or if you
have a
sensor that is failing a little bit then
you probably say okay
my led is not more anymore okay with
rain if it starts raining
because i have this problem i need to
stop so
these are all uh relevant uh things that
we need to touch upon and
and this is a great comment because i i
really think that internal state and
internal things are really
need to be part of the odd the industry
the audience the community is still
fragmented on this opinion and
last point i want to mention this is
also which is another topic recurrent in
in my standards is
the behaviors our behaviors part of the
odd
so behaviors are still
by behaviors i mean other agents
behaviors
i think so because for example
i can have and and data that is
collected by autonomous vehicles
is also behaviors like i can go around
san francisco
and build uh let's say a behavior
map with all the trajectories that
dynamic agents perform over time in all
my runs that
i see through i i passed through this
street in san francisco
a mill like a thousand times i i have i
collect the trajectories and i put it
there i can build a statistical model
that says
in this piece of the world these
behaviors are way more common than in
others so
i think well should this be part of my
odd
probably because i need to figure out
whether my system is
trained well enough to be to to deal
with jay walking homeless people in
downtown san francisco or not this is od
right i mean it's a statistical idd it's
a
kind of a probabilistic led which is
also another topic we are discussing but
it's part of it you you want to have
probably a lot of
pedestrian homeless jaywalking poor
people in
downtown uh i don't know in a small town
in
in in northern italy right you'll have
them
in san francisco a lot i can guarantee
you so you need to
you know be able to specify that as a
specification of the system that you are
able to deal with that
thank you very much
i think uh i can speak for myself and
the others it was really interesting
nico and we will follow up
with another uh chat i know you are also
busy and we also have another meeting so
i want to thank everybody who joined and
you especially uh thanks again and uh
enjoy the sun also for us thank you
thank you i will
yeah and i hope to talk to you soon yeah
okay thanks everyone and
feel free to reach out to my emails
nikoi deepan ai
or linkedin works thank you bye
bye
you


What's Up With Bard? 9 Examples + 6 Reasons Google Fell Behind [ft. Muse, Med-PaLM 2 and more]

this video was supposed to be about the
nine best prompts that you could use
with Google's newly released Bard model
it's just
like problem every time I tried one of
these epic ideas gpt4 did it better I
really wanted to come out here and say
look you can use it for this or for this
as you'll see it just didn't work out
that way so instead reluctantly I had to
change the title now unfortunately it's
just a comparison showing how much
better gpt4 is compared to Bard a lot of
people wanted this comparison after my
last video used Bing for comparison this
one's going to use open ai's gpt4 but I
wasn't satisfied with just showing you
the problems with Bard I wanted to find
the explanation in the end I didn't find
one reason I found six as the why Bard
is so far behind and why Google is
losing the AI race let's get to the
comparison first one is coding and as
you can see Bard refuses to do coding
they actually mentioned this in the FAQ
that Bard won't do coding for you as it
says I'm designed solely to process and
generate text as you can see it's a
fairly basic coding Challenge and Bard
won't do it gpt4 had no such qualms and
the code worked first time of course I
did check it and it worked but this was
just a simple challenge to turn letters
into numbers next and even worse for
Bard it can't summarize PDFs this is
going to be such a common use case for
Bing using gpt4 by the way it didn't
admit that it couldn't summarize the PDF
it summarized a completely different PDF
and if you check the other drafts none
of them summarize the correct PDF of
course the gpd4 accessed via open AI
also can't do this because it can't
access the web it also picked a
completely different paper but our old
friend Bing could indeed read the PDF
and summarize it okay what about
summarization when I literally paste in
the text that I needed to summarize this
is surely the most obvious use case of a
language model imagine you want to
summarize a meeting via Google meets or
shorten an email thread in Gmail it has
to get this right I pasted in the same
New York Times article into Bard and GP
for and I am sad to say that Bard
fluffed its lines the link to the
article will be in the description but I
have read it carefully and it makes
numerous mistakes let me scroll down and
show you this erroneous summary first it
says the FED is expected to raise
interest rates but doesn't say by whom
second it starts chatting about Full
Employment and inflation not only is
full employment not mentioned in the
article at all it also gets both numbers
wrong the unemployment rate in America
isn't currently 3.8 and inflation isn't
at 7.9 I check these against the latest
data and you can check it yourself but
both are wrong Bard also keeps going on
tangents like stocks are typically
considered to be risky Investments than
bonds okay that's fine but why are you
teaching me Financial advice it was
supposed to be summarizing an article
honestly it was a pretty unusable
summary so bad that to be honest you'd
have been better off just not reading it
trust me I am not an open AI Fanboy but
its model is just better currently
notice how a summary it doesn't go on
tangents and it clarifies that it's
investors who think that there will be a
quarter point increase the five bullet
points are succinct and accurate this is
a pretty colossal loss for Bard what
about light content creation and idea
generation surely it could do well here
just something innocent like create
eight new YouTube video ideas with
titles and synopses on integrating
generative AI into retail if Bard can't
be used by analysts maybe it can be used
by content creators not really I mean
you make your own mind up but these
titles are pretty repetitive and Bland I
know I can't really complain because my
channel name is AI explained but these
titles are just unoriginal and the
synopsis lack detail I'll let you read
these but compare them to Gypsy 4's
outputs each title is different and the
ideas are much more explored and nuanced
okay fine what about email composition
and I have to say count me a skeptic on
this one I have never ever found that
any model let alone Bard can do a decent
job at this it's not always that the
emails are bad it's just that the time
it takes me to teach the model what I
want to say in my email I could have
just written the email I'm going to make
a prediction at this point I don't think
using language models to do emails is
going to become that common of course
feel free to quote me on this in a
year's time now you're probably thinking
I'm being harsh this is a perfectly fine
email I did leave a thumbs up it's just
that I would never use Bard for this
purpose and I would also never use gpt4
like I don't want it to make up all
these extra details about what I'm going
to discuss with John it's just too risky
to send an email that has any chance of
hallucinations I know you guys might
think that I really love Bing but it's
even worse here it claims that I've
added relevant data and graphs no I
haven't I never mentioned anything about
data and graphs now my boss thinks I'm
going to do data and graphs what are you
doing Bing and then you're going to say
why am I using creative mode well if we
use balance mode or precise mode we go
back to the bad problem it's an okay
email now but look at the length of it I
could have just written it out would
have been quicker to do the email than
the prompt I was beginning to lose hope
in Bard so I tried writing assistance I
picked a paragraph that someone I know
used for a personal statement to get
into University of course they were
happy for me to share it it's decently
written but could be improved
significantly I asked Bard rewrite this
paragraph with better English make it
original professional and impactful now
Bard did remove some of the errors but
it again went on a wild tangent trying
to sell a career in data science as if
we were some sort of recruiter now I'm
not going to be too harsh if you just
take the first paragraph it's okay GPT
4's output is better but still has some
problems now I think some of you are
going to laugh at what happened with
Bing it simply refused to do it twice I
pretty much had to trick Bing to get it
to rewrite this paragraph first it says
my mistake I can't give a response to
that right now I tried again it said hmm
let's try a different topic sorry about
that that finally I just asked the exact
same thing with different words I said
rephrase this text with smoother
language it seemed to like that and then
did the job I think it's the best output
but still has problems anyway this is
not a grammar lesson so let's move to
science and physics and Bard completely
flops it gets this fairly basic physics
question wrong so how can it be a tutor
for us for a student to effectively
learn from a tutor there has to be a
degree of trust that the tutor is
telling the truth gpt4 by the way gets
this one right I even asked Bard to come
up with a multiple choice quiz it
definitely came up with the quiz problem
is quite a few of the answers were wrong
I didn't check all of them but look at
number seven and number eight the
correct answer just isn't there gpt4
does a lot better with really
interesting questions in increasing
order of difficulty now it does have a
some slip UPS look at question four
there are two correct answers one is a
half one is five over ten but they both
simplify to the same thing gpt4 was also
able to give these explanations I do
think the day of AI tutoring is fast
approaching I just don't think it's
quite here yet and certainly not with
Bard I think the point is pretty much
proven now so let's move on to the
explanations why has Google fallen so
far behind first a lot of its top
researchers have left there were eight
co-authors at Google for the famous
attention is all you need paper on the
Transformer architecture that's amazing
right they pretty much invented
Transformers problem is now all but one
of the papers eight co-authors have left
one joined openai and others have
started their own companies some of
which I'll be covering in future videos
speaking of which if you're learning
anything from this video please don't
forget to leave a like and a comment
next potential reason is that they don't
seem to want to interfere with their
lucrative search model as the product
lead for bad said I just want to be very
clear Bard is not search if you haven't
seen my initial review of Bard which
pretty much proves that it's terrible at
search do check out after this video If
Bard is designed for search what is it
designed for as the article points out
they haven't really provided specific
use cases next are they worried about
safety and accelerationism or are they
looking to buy up a competitor to open
AI they invested over 300 million
dollars in anthropic the stated goal of
that company is to work on AI safety and
Alignment so is Google trying to be on
the right side of history and place all
of its bets on safe AI or are they
trying to do to anthropic what Microsoft
did to open AI itself I'll be following
this particular story quite closely over
the coming weeks and months next maybe
Google has better models that they
genuinely don't want to release because
they fear a PR backlash they had the
Imogen text to image model that was
better than Dali 2 and they didn't
release it Google said it was because
Imogen encoded harmful stereotypes and
representations I dug into the original
image and paper and it was indeed much
better than Dali 2. Google wasn't
bluffing they had a better model and
that wasn't the last time in January of
this year they released a paper on Muse
a text to image Transformer that was
better than both Imogen and darli 2. in
case anyone thinks they're lying here I
think is the proof The Muse model
outputs are on the right the image and
outputs were in the middle and open ai's
Dali 2 outputs are on the left strikes
me that Google's Muse is one of the
first models to get text right
mid-journey even mid Journey version 5
definitely can't do this so why didn't
Google release this well I read to the
end of the newspaper and they say this
it's well known that models like
mid-journey and Muse can be leveraged
for misinformation harassment and
various types of Social and cultural
biases due to these important
considerations we opt not to release
code or a public demo at this point in
time let me know what you think in the
comments but I think it's more than
possible that Google has a language
model that's far better than Bard and
even far better than Palm perhaps
leveraging deepmind's chinchilla model
and that they are genuinely keeping it
back and not publishing papers on it
because they worry about these kind of
considerations anyway I do have a final
theory about Bard and that theory is
that they might have been working on
what they regard to be more serious
models in December Google released this
paper on medpalm it's a language model
tailored to help in a medical setting
and if you think it's accuracy of 67.6
in answering medical questions was good
wait till we hear about the fact they've
now released medpalm 2. here is a
snippet of Google's presentation on
medpalm 2 released just a week ago today
we're announcing results from medpalm 2
our new and improved model
mad Palm 2 has reached 85 accuracy on
the medical exam Benchmark in research
this performance is on par with expert
test takers it far exceeds the passing
score and it's an 18 leap over our own
state of art results from medpalm
medpumm 2 also performed impressively on
Indian medical exams and it's the first
AI system to exceed the passing score on
those challenging questions but finally
what does this say about the near-term
future of Bard well the more users a
model gets the more data it gets and so
the more easily a model can be improved
as this Forbes article points out
Microsoft now has access to the valuable
training data that these products
generate which is a dangerous Prospect
for an incumbent like Google and it's
not like Google doesn't know this the
CEO of Google admitted that products
like this talking about Bard get better
the more people use them it's a virtuous
cycle but does that mean that it will be
a vicious cycle if everyone uses gpt4
instead of Bard with less data does that
mean there'll be less Improvement of
Google's model only time will tell and I
will be there to test it thank you very
much for watching and do have a
wonderful day


Responsibility in the age of AI  (Jeroen van den Hoven) - 1st AiTech Symposium

I'm very much looking forward to that I
will together with Filippo set the scene
a little bit the bigger picture if you
want how we arrived at this this topic
and and this this approach that you know
there's sketched out I would take at
least 50 minutes to mention all the
things that are relevant that I cannot
do but we split the presentation in two
parts so Filippo will go into this
meaningful human control and talk about
how it is applied in a project that they
do as a Dutch Research Council project
on meaningful human control and assisted
driving autonomous vehicles and so you
can see the the concept in action if you
if you want and it started we feel a
little bit responsible because we took
this notion of meaningful human control
out of discussions that started three
years ago and we participated in people
and I in Geneva when it was about
meaningful human control over lethal
autonomous weapon systems and we noticed
that the the the diplomats and the
international lawyers were all talking
about meaningful human control and
they're they're very happy about you
know this term that they came up with
this term but then we coming from a
Technical University we thought about we
have to explain this to our engineer
colleagues in Delft so how are we going
to do that and that prompted us to write
a paper on a conceptual analysis that
provided the conceptual analysis of this
term and Felipa will give you the gist
of that so what I will do is present you
the bigger picture in three steps some
under 20 minutes the ethical problems
and how I lump them together or cluster
them and then a little bit about how to
do epics but if Gainey will also go into
more detail later on
so first the bigger picture but first I
need a glass of water because I'm
suffering from a cold can what's over
there okay good wonderful thank you so
much yeah
the bigger picture really quickly hmm
I'm also advising European Commission
has done something strange there but
anyway they think all think that a eyes
important so it's and and you can find
many many quotes it's gonna change our
lives this guy also thinks if you get
dominance in AI you will rule the world
in China they also think that that's the
case and they're using it this guy's
using it in Silicon Valley's using it in
a completely different way so a short
terms there's a didja battle for digital
supremacy going on and in Brussels the
question is where is Europe going where
does it leave us and is Europe going to
be the Museum of the world nice to visit
and take pictures of or is it going to
be the cradle of a new enlightenment it
could well be well the Commission has
said never mind China and the US in
there can relaxing the you know the the
laws on use of data or privacy and
training up there they're deep deep
learning algorithms let's just stick
with our core values and let's focus on
that and we know exactly what they what
these are is the Charter of Fundamental
Rights and the Convention on Human
Rights and have gained II will go in
into detail in how we can design for
them so my call actually made an
interesting suggestion three years ago
there's a third way there's a European
approach that is true to these
constitutive legal binding treaties that
make you know the put our human rights
and ethical principles cor-ai for
Humanity is Brussels talks about
systemic rivals so these are no longer
ideologies these are coherent clusters
of policies infrastructure monetary
policy military strategy and all powered
by AI and Big Data so you have the u.s.
Silicon Valley model innovate fast in
the gray zone break things first and
apologize later minimal regulation
Chicago school economics libertarian
values hard power and Homo economicus in
strong individualist and then you have
China and that is socialism with China
characteristics social harmony party
rule sharp power and a behaviorist
utilitarian collectivist kind of view of
man and Europe as I just explained a
Kantian conception of the person all as
autonomous free and a responsible agent
and so we want to in Europe
russell's wants to hang on to that
conception so that's the that's the
that's so the good the discussion about
a good digital society is about this it
is about a model of man and it's about a
model of society and that is what is at
stake so if we are going to discuss
meaningful human control to a large
extent we try to salvage this core idea
so what are these ethical problems now
of course it goes without saying that
these are all important safety loss of
lives health well-being it's a no
brainer of course we we have to make
sure that the robots that we produce out
of our labs don't kill people and don't
harm people but I want to divvy up the
ethical problems in it's like a
different way in in in a way that
provides a certain coherence to them and
relates them to the core problem that
I've just sketched out for Europe for
the world namely this conception of a
responsible free agent that can be held
responsible that can feel responsible
and for which it is fair to hold that
person responsible now you know that
these are conditions necessary
conditions that need to be fulfilled for
someone to be responsible and you can
find out by saying okay suppose that you
accuse or suggest that someone is
responsible someone could say oh well
sorry I didn't know or sorry I wasn't in
control or sorry I had no choice or
sorry I I wasn't my good self
I did something because I thought other
people expected this of me or I what I
wanted to prove people wrong so I did
not on my own accord but I took my cues
from a social environment because that
people were looking over my shoulder
right
so all of these conditions undermine
this idea of my suggestion is that AI
kind of under could undermine all of
these conditions for moral
responsibility so it's a big threat to a
core idea of Western societies so I'll
just you know briefly go through them we
have made things extremely complex and
it will come back in many discussions
today as a result of introducing big
data and machine learning it's become a
little bit of a black box Society and we
use it everywhere it affects people in
crucial ways so introduced possibly
machine bias you know the the negative
versions of it or good versions we don't
know as long as we don't know we cannot
justify what we have done we don't know
exactly what we're doing a nice paper
has machine learning become alchemy well
in a in a certain way it has become
alchemy the first AI legislation is
already in place and it it addresses
this problem its article 22 of the GD P
R and that if you if you apply
algorithms or machine learning to people
and it affects them in a big way then
you have to be able to explain what you
did
alright so that's that's and and some
additional requirements so some people
think that AI itself could provide the
solutions to this problem of
transparency and the lack of knowledge
which chips away at one of the
fundamental conditions for
responsibility right so we have to
design for knowledge in such a way that
we restore individuals both the
designers and the users to positions of
responsibility
so all kinds of temps are made you could
try to torture the system in such a way
that it gives you the information that
you need in order to provide an account
or a justification you could do these
kind of things just so so in identifying
a zebra you know where it has the system
been looking at so that you can account
for it hmm very interesting work of
Judea pearl also trying to kind of
bludgeon these machine learnings that
this
systems in such a way and torture them
too that they give up their their
secrets and and and allow you to say
something about the causal mechanisms
that are underlying this work on
algorithmic recourse so you explain what
the algorithms are to the people who are
affected by it in such a way that you
that that you provide opportunities for
the people affected to temper with the
algorithm so if I would do this oh then
I would get me I would be selected for
this procedure or I would be admitted to
Delft University so perhaps I should
spend two more weeks on you know Crick
are doing scoring better on my TOEFL
test or something like that
so and all of these attempts these
engineering attempts are made to do what
to be able to say yes we had the
adequate knowledge of what we were doing
so to restore this knowledge condition
then this condition of control oh sorry
I wasn't in control right the quest for
meaningful human control and filippo
we'll talk about that so I will skip
over that but we have seen the examples
of how this could really be a problem
especially when you start to apply to
lethal autonomous weapon systems yeah
you like this target you may also like
that target you know as the recommender
system especially recommended for you we
don't want these things right people to
throw up their their hands and and not
be be accountable because they they were
not in control or they didn't know what
was happening right so they they have
plausible deniability because AI has
helped them to deny that they are
responsible so this is what Felipa will
talk about so that we we need to restore
ourselves to this position yes we had
the kind of control the kind of control
that is a necessary condition to take
responsibility freedom and choice I see
a colleague cost per course and it's
very much related to your work and AI
could be a threat of undermining this
condition for responsibility if you're
sorry there was nothing to choose or I
wasn't aware that I could choose right
so we people are locked up in these
filter bubbles it's sufficiently known
now
but this is a big threat if you have big
data you have machine learning and you
have advanced behavioral signs then
freedom to choose is becoming a little
bit of a problem and this is one that
you know very well but other people may
not have seen and I think it drives home
the message very very clearly you can
subscribe to The Economist for $59 or
you can subscribe to the print edition
so this is the online form 59 this is
the print 425 or special offer both 425
right so everyone you're a good company
if you say well third option well that's
that's let's do both here is an
alternative choice architecture that is
that it's not you know just an accident
that it's there the economy is online
459 or the print and the web 425 now
everyone says oh well let's forget about
the paperwork let's do just the online
subscription you see that the way you
line up these alternatives you you you
engineer the choice set prompts you to
pick this one why this is the outcome of
advanced behavioral research a lot of
big data on how people choose and a lot
of pattern discovery in how what people
what how people are motivated and what
they're what makes them tick right so if
you have that you are sitting duck
first these people as customer knows
very well and this is going to be a
problem for that third condition of
responsibility freedom of choice right
so we had knowledge we had control we
have freedom of choice and AI is
chipping away possibly undermining all
of those conditions that's that's the
idea and look at what Sunstein the
nudging Pope says himself it is possible
that companies that provide clear simple
products would do poorly in the
marketplace because they are not taking
advantage of people's propensity to
blunder that is what is at stake and
they are greatly helped by big data a
machine
learning a I is their game and in a way
that we don't have access to it and of
course it has been used also in the
political realm as it's become clear
here I couldn't find a nicer picture of
this guy but you know he this is this
this is what has been at stake micro
targeting and manipulation of choice
sets so we wrote this paper will
democracy survive big data and
artificial intelligence in Scientific
American we have to be able to say truly
yes we made our choices freely and we
stand by them privacy it's a slightly
atypical way of involving privacy here
but I think it can work
same thing here big data and machine
learning AI combination very powerful
yes we know who you are what you where
you have been and more or less what you
think and we need to take this very
seriously we have machine learning
reconstructing what people have been
looking at without knowing what you have
been looking at actually just by
analyzing your brainwaves so this is
what people have been looking at and
this is what the system kind of does on
the basis of the analysis of their neuro
activity so so that's that's already
quite interesting right and we know
these kind of studies you know from 300
likes the system can better predict what
you will do on a psychological test and
your partner so that is so doing what
others expect us to do or to be right
becomes a threat to our freedom and the
the freedom that we require to say yes I
own this decision I did this on my own
accord and not because people were
looking over my shoulder and expected me
to be this or to do that and this is one
of the important elements or ingredients
in privacy so that we can say yes my
choices were mine I made them freely
that were under my control and I know
exactly what I have been doing and
therefore I am responsible
I'm feeling responsible I can take
responsibility you can hold me
responsible so that is what is its take
we're designing a society in all the
applications that that tries to optimize
these conditions these moral conditions
so that we salvage the idea of moral
responsibility on which our whole
society is built our legal institutions
our social institutions all built on
this very core idea
not necessarily in China not necessarily
in Russia they may be willing to abandon
those ideas for the moment I have the
impression that we are not so let's try
to tick those boxes in our designs now
this is an interesting thing you know
this of course you know the trolley
problem we're sick and tired of it we've
seen so many of these examples but the
interesting thing is is if you use this
and teach this and discuss it with
students in Delft they say oh well shoot
this guy you know do this or that and
then you you bring your account or your
utilitarian theory to it and we go into
an analysis but engineers say that
design that design because the problem
that this poor guy has is an is a
function of the design he can only do a
and B just pull the lever or not right
so he may have wanted to do something
else but it was not designed in never
you will find this in a philosophical
analysis because it's of course in a
philosophy seminar room it's a stupid
question you're asked to leave
immediately because it's a thought
experiment right you're not supposed to
temper with the thought experiment
because it's there to make give you a
hard time but if you were thinking about
the real world then the engineering
remark is a very good one because we
want the world to be safer we want the
world to not if possible have dilemmas
and create moral problems for operators
we want to prevent that so we'll have to
look at the design history
of how did who designed which idiot
designed it in this way right so this is
the question we have a responsibility
for the responsibility of others this is
a second-order responsibility we have
responsibility for the responsibility of
others now how do we do ethics in the
age of AI probably running of time and
Afghani will go into more details so I'm
very comfortable doing this very fast it
is by design and I think all the
examples have shown clearly that is you
know you can you can design a bench in
this way you don't have to chase away
hire 20 people to chase away people if
you don't want to sleep you know them to
sleep on it
so you can design these things in it's a
controversial example you need to have a
debate whether that is solving our
problem we perhaps we need to solve the
problem in a different way than kind of
building these kind of benches but this
is the key problem of value sensitive
design this is a bunch of value societal
moral requirements there are
non-functional requirements and they
have to be somehow not like the iron
rods in the bank but they need to be
built into over everything we do in the
algorithms in our our coding every every
detail of a system and we have to be
able to explain that we did a good job
so that's the that's the idea we do
design for X designing for all of these
values by breaking them down from a very
abstract level and until they hit design
requirements that our colleague said the
technical units can can work with so
these kind of things privacy by design
is a good example if you want to count
the number of people in this in this in
this audience but you don't want to give
away their identities you can do this
it's a very simple example of how you
can design in privacy and at the same
time have the functionality that you
want out of the system
it's a simple thing and of course it has
given rise to a whole lot of lot of
technical solutions coarse graining the
data Kay anonymity differential privacy
homomorphic encryption Enoch knows about
this privacy preserving machine learning
etc so these are attempts
to hold on to these ethical values
design the mean in such a way that we
make good use of all the AI without
giving up on all the and the same needs
to be done for the conditions that I've
just lined up and discussed on
responsibilities and I think it can be
done the fascinating thing is that we
have a wonderful team and we have you
here to discuss this with and very much
looking forward to that without further
ado Filippo who will tell you more about
meaningful human control thank you
[Applause]