AiTech Agora: Lotje Siffels & Iris Muis - Zeitgeist and data: the danger of innovation

uh
lucia is i uh give the floor to you all
right
thank you so much um i will start
sharing our slides first of all
uh welcome today we're very pleased to
be invited to this a uh
how do i say it igora yes agora meeting
um
really nice to uh be able to e-meet all
of you and discuss with you the topics
of today in the second part of this uh
meeting
um i took the liberty of inviting some
colleagues from utrecht university so
you will see some unfamiliar faces today
um
but um first of all let's uh start by um
uh introducing ourselves
uh loki please go ahead
yeah thanks again indeed for the invite
it's great to be here um
so i'm lucia sifuls i am now a phd
candidate at the rutgers university
uh working on a project about digital
good which is mostly about the
digitization of healthcare and also the
influence of big tech in healthcare
and before i started that about i worked
with edis at interact university
on dida which is a tool she will explain
more about
today i think for now that's enough
things
all right so my name is i still work at
utah university um within a team called
utrecht daily school
and we researched the impacts of
datification on society so we try to
really bring that humanities perspective
to
to tech
[Applause]
and
you know within that perspective data
ethics is of course very uh very
important so that's one strand of
research we are very interested in and
we have focused on for the past five
years
so or six years even
and when luchi and i uh worked together
a couple of years ago
data ethics was was our joint focus so
today we would really like to share with
you our experiences
in working with data ethics ethics in
external organizations because we
really did a lot of work guiding data
ethical sessions
mostly within government organizations
and you know
after doing this for a couple of years
and and within multiple organizations um
big organizations smaller organizations
we
started seeing um
recurring
arguments within the ethical debates
patterns
and
we would like to share with you some of
these
recurring themes and we in the second
part of this uh this workshop we would
really like to hear
you know your experiences or or opinions
about this and really turn this into a
discussion
um
uh
which could be fruitful for further
research
um
so first of all what exactly did we do
um
so
we worked with
with an instrument called the data
ethics decision aid or dda
for short
which was created in 2016 by italian
school to
guide organizations or project teams
specifically
in
developing ethical algorithms ethical
data projects in general so really
coming from a
public values perspective
this
deliberational framework
was meant to guide project teams in
really operationalizing public values
within their specific data projects
so on the slide you see what that looks
like
there's a big poster
and the project team is standing around
this poster
they are answering all sorts of
different questions
surrounding different topics
which range from technical topics like
anonymization and data sources used
access to data and security
to
topics such as privacy or bias
so all of these questions you can see
there on the poster and the project team
is meant to deliberate with each other
about these questions and to really come
to
decisions about how to best adopt these
you know different public values within
their project design
or algorithm design
so that's just to give you an idea of of
what it looks like and
um for these
workshops which typically take
around three hours so quite long but
we really feel that like that's that's
necessary for a good ethical reflection
you know it takes some time
to get to the core of things
we really advise project teams to to
invite people with a range of different
backgrounds to these sessions so people
with a background in tech
people with a background in policy
development or law
communication even
so to really
stimulate different viewpoints being
brought into these ethical deliberations
um
this is the poster the english version
of the poster the i i realized the the
letters are very small so i don't expect
you to be able to read this but um
if you want to take a closer look just
let me know and i will put the link to
all of the downloads of dita in the
chats later on
um
we also
during the past two years started
working with an online version which you
can see here
so we would
not no longer go to organizations but
have these kinds of you know teams
meetings with uh people from
municipalities people from uh from other
governmental and even sometimes
commercial organizations
in which they could fill out this pdf
version of dda
and you can see here at step one we
start out with
defining the values of the organization
so you here you really see that public
values perspective
reflected in the design of and the
process of data
so you know after five years of of
ethical deliberation what did we gain
out of this and what were the uh the the
things we saw um um uh
reflected in in almost every
workshop uh so so first of all just to
give you an idea of of the scope
we did these kind of sessions within
more than 80 organizations
most of them dutch organizations couple
of them
german or or
belgian organizations but like 95 was
was dutch
most of which were local government
institutions so municipalities
um the cases that were brought in were
very different so it could be a more
complex
complex project like a project with
image recognition or risk prediction
to
more simple projects like combining a
couple of different data sets
the project teams were interdisciplinary
people with different backgrounds and
you know every
every organization obviously has
different organizational values
which also cost a different outcome of
each workshop so of course the
organizational values are also very much
dependent on the
political color of the city council
when talking about municipalities so
more liberal
municipality
has different organizational values than
uh
uh left uh city council for instance
um
but most importantly uh we got lots of
data
out of these ethical deliberations
because for us uh dita mainly acts as a
as a research tool
uh because as i said before we're very
interested in the impact of datification
on society and also
uh the impact of of of the way a city is
governed for instance so
dida really uh gave us an entry point
into an organization
to have a seat at the table
not as
researcher with a survey for instance
but really as as expert or as
as moderator
uh which which gave us
you know very valuable insights and and
and very intimate details sometimes of
the impact datification really has
within an organization so here you see
really that the duality of data so for
the external organization it's it's an
impact assessment which can
guide them into ethically implementing
data projects and for us as researchers
it really acts as a as a research tool
which gives us
insights into the way datification
shapes society
if you have any questions
by the way feel free to to ask them at
the end of this session i would really
like to discuss with you also
uh
the use of data as a research tool
uh it's something where we're very
interested in hearing from from you also
how you collect your data
and and your experiences with these
types of of instruments
so um
now we get to the juicy part
what did we observe
and i would like to give the the word to
logi for this
thanks edis so yeah for me
i am allowed to do the juicy part it's
really nice
so um i first want to just briefly
discuss
uh some general observations that we
made uh
and then go into uh the subject that we
really want to discuss today which we
called the zeitgeist narrative
um
so
also just to give you an idea of the
kinds of things that we would see and
that would already be very interesting
so first of all
there was a huge difference between
different kind of organizations in what
kind of expertise they had
this expertise we mean with that
technical expertise but also expertise
in the practice of the field so these
would be local government institutions
doing data projects about a given thing
for example
how to deal with citizens in debts or
how to prevent
their citizens from falling into debt
and it would differ greatly what kind of
expertise would be at the table at these
ethical deliberations and when there was
somebody with expertise from the
practice of the field
in this case somebody who actually
worked with spoke with people who were
in debt debts
who knew the practice of it
that would make a huge difference in the
ethical deliberation so it's already
interesting that this varied so greatly
but it's also interesting that we could
see in the ethical discussion how
this changed the ethical discussion so
values like
equality dignity
like the absence of bias privacy these
are very abstract values when they're
just there on the list but when you have
somebody there who knows how these
values work in practice
then you can get a real ethical
discussion so this was really necessary
for the ethical discussion and the same
goes for the second point so this is
about the more technical expert
expertise uh the data literacy of the
people involved and this also differed
greatly per uh government organization
that we worked with but we saw how much
of a difference it makes when you have
people on the table who know
the nitty-gritty of the technical
aspects because as i guess most of you
here will realize
these ethical values
they are ingrained in the nitty-gritty
of the technical aspects and if you
don't even know
um a simple example is like if you don't
even know what anonymization is or
pseudonymization how are you supposed to
talk about these abstract values
and so we saw ethical discussions having
such
being so much more effective if the
people with uh the technical expertise
were at the table
but oftentimes this kind of expertise is
not there in project teams because it's
being outsourced so not all local
government institutions
have this expertise within their
organizations and they hire external
parties to deal with that part and we
could really see how much of a
difference this makes because this means
that
yeah most of these projects teams
couldn't really discuss
um the ethical aspects of the project
they were developing
and most of the time these external
parties they weren't there for the
discussion right so uh the whole
a whole part of the ethical decision
making was also outsourced in this case
and this is a war something that worried
us that we could see
but it was very interesting also to see
how it works
um and this also uh um yeah has to do
with the third point that uh when
there is a lack of either kinds of these
expertise like the practical or the
technical then uh yeah you run the risk
of having
responsibility gaps
if if people don't really know how it
works
uh when you ask them well who is
responsible for this aspect or who is
responsible when something goes wrong
what what will you do when something
goes wrong
um they won't know
and especially this is of course
specifically for data projects a very
prominent thing because people don't
really understand where the decision
making even lies when there's all these
technical things involved or when
there's an external party involved
so this was also something we saw quite
regularly
and that was dependent on the kind of
expertise at the table
but a more positive notes uh the fourth
thing is that we also notice that civil
servants are really good at having
ethical discussion they are well
equipped to talk about the common good
so there is something in in our uh
structure of our local governance where
this this is this works out well these
civil servants
they are very good at articulating
values and also thinking about the
common good for their citizens
we know this also because we were able
to compare it with doing these workshops
with some commercial organizations and
this was a big difference uh mostly the
participants from commercial
organizations
they were not used to thinking about the
public good or public values at all
so
we suddenly
noticed that we had to start at a whole
different level of having an ethical
discussion
and then
um fifth point which gets a bit more to
what we i will discuss after this a bit
more elaborately um
one of the things we really noticed is
that when ethics became a box to be
ticked
uh it loses its value and what we mean
is that we notice a big difference
between workshops where participants uh
usually out of their own interest for
the ethical aspects of the projects
would want to do a kind of ethical
deliberative process and ethical
assessment
and the workshops where
um the municipality had uh obliged them
to do an ethical assessment so they had
to just take this box of yeah we did an
ethical assessment if that was the
mindset of the participants then you
cannot really have an ethical discussion
um they just want to be done with it
um
and then finally uh the thing i will say
i will go into
deeper now is the zeitgeist apology
um so
the zeitgeist narrative or apology so we
call it an apology when it's used
within things
the um within the discussion as kind of
a justification for doing a project
without really thinking about the
ethical considerations
what we mean with that is
any time we notice that participants
said things like these we just have to
get through
you just need to do this everybody does
it
um
one participants during one of the
workshops actually called data projects
toys for the boys uh which i thought was
very interesting i will go into
it later a bit deeper
but they also sometimes called it a
system jump
and what they mean is like um
it
is required to just make this switch to
go on into this new system this this new
future that there is and and
it it will happen anyway and it needs to
happen anyway so
why think about it too much or all these
ethical considerations
are then not taken seriously
so it is really
about um
anytime a participant did not seriously
enter into the ethical discussion
because they thought something like a
data project was inevitable because of
progress this was the future and you
cannot
deny the future that is coming anyway
and more than that it is also
bad to try to prevent this future from
happening because they really see
uh technological advancements in the
form of these data projects as as a
high-speed train
that is going really fast and you have
to get on it because otherwise you miss
it and you're left behind
and this is then one of the worst thing
that things that could happen
now why did we think this was so
interesting well it's because this
zeitgeist narrative
it has a huge maybe even bigger than the
initial observations that i mentioned
influence on the quality of the ethical
discussion we had
with these participants um
and this is because
uh of
a couple of reasons a couple of
characteristics of this narrative so
again as i said progress is seen as an
inevitability
which of course it makes you a bit yeah
it makes no sense to really carefully
consider something that's inevitable
anyway
um
but also valuing innovation
so this progress and innovation in
itself is just put
like on a higher level than the other
values and like i said these civil
servants are pretty well equipped to
think about public values and also to
recognize like there's a plurality of
values and we we want to consider them
we do not want to place one on top of
the other but then when it was about
innovation or progress or this idea of
trying not to miss the train this was
really forgotten and it was like you
could use the argument of innovation
to
trump any other value that was relevant
in the project
so
innovation is a way to
temporarily waive other ethical
considerations
and then the for it the third point is
that it invokes a sense of haste
um which also of course is in the way of
a very of having a careful process
where you
think about ethical values
like it said you need a long time to do
it and a lot of things will come out
that will slow down the project
and this doesn't fit with this feeling
of haste that this zeitgeist narrative
invokes you have to get on it now
otherwise
we will be left behind
um
and then the fourth point is
uh i guess
for us this was the most uh
important one that it invokes this sense
of powerlessness and this is because
yeah participants they would sometimes
really be downcast during these
workshops because they would be just
like well it's going to happen anyway
what are we doing here
and they felt so powerless because
sometimes they were actually interested
in the public values and they were
interested in all the ethical aspects
and they were worried and they had all
these great insights about possible
problems
with these data projects but they just
felt like they were powerless to stop
the
advancement of these data projects
and so
um this also leads to another aspect
that is that sometimes the participants
so the civil servants themselves
were really thinking in this tight guy's
narrative and they uh
uh they weren't very critical about them
but sometimes they were very critical
about it and it wasn't really that they
themselves felt like progress
was this train they had to get on but
they knew that um the politicians that
at that moment were determining which
projects needed to be developed that
those were thinking along those lines so
they felt like yeah we can be here
thinking about all the ethical aspects
and we can even tell them that this is
problematic but they won't listen anyway
they will want to do it anyway they will
want to get on this train
um and this is some of the participants
they
mentioned things like this
and usually this was like way at the end
of the workshop so after having a two
three hour discussion about all the
relevant ethical aspects a really rich
discussion
and then at the end one of the
participants would say like
but it's going to happen anyway
and then if you ask why is it going to
happen and they say well it's to score
politically the political image is the
municipality's one number one priority
so they want to score with data projects
um and another set
this is another quote an ethical dilemma
is the output of the project so all the
ethical considerations that we had
discussed for three hours at a time
versus just the eagerness to start the
project so this eagerness is just
trumping
all the careful considerations that you
could have about these
and um
yeah so um for us it was really the
sense of powerlessness that we could we
could find we could see and we saw how
pernicious it was because it just
precluded all the ethical considerations
that were being discussed
um and also and this is why i put the
second sentence here um
it is not just the ethical deliberation
but it also does something pernicious to
the relationship between
um the civil servant and their citizens
and this is also something interesting
so with data projects more than we think
with other projects civil servants will
have
an urge to think think like yeah but
people will get angry about this anyway
because they are data projects
and a civil society will get angry about
it anyway because their data projects
but we have to get through it like we
have to do it still because otherwise we
will miss this train so this whole idea
of that public opinion uh uh that there
may be
a richness to a democratic process that
the civil servants were actually usually
very
aware of in the case of data projects
sometimes this was just gone
and um this also leads to
the final thing that i wanted to say
about this because we want to leave a
lot of room for discussion
but
we haven't really worked out yet what
kind of conceptual frameworks we could
use to think about this i mean we called
it a zeitgeist narrative or apology
because that's
it resonated with what we what we felt
and saw but there's of course a lot of
ways that this has been noticed before
by other scholars
um but linking to this uh preclusion
also of the democratic aspects
of these um
um
yeah the the the consequence it has for
thinking about democracy and about uh
relationship between
uh the civil servants and the citizens
is that it it yeah it's a kind of
technocratic way of thinking so it it
gets you out of thinking about
democratic values and get you into but
the system
will know what's best and we know what's
best because this is progress so we know
better than our citizens what's best so
it doesn't matter whether they get angry
uh because there's
there's no sense anymore that there's
value in the democratic process itself
so it's it's kind of technocratic way of
thinking
and i just want to briefly hide that
because i don't have time to explain
uh
the difficult frameworks of i don't know
if any of you are familiar with uh
boltonski and davino or botansky and
geopalo but
i've been working with these frameworks
and it
it really shows how for civil servants
that are usually discussing things among
themselves in a kind of civic logic
which is for them very relevant so it's
about democratic values about processes
of democracy
uh it's about
rights equality
but when it's about data projects
suddenly this logic shifts to a
different kind of logic which you could
call an industrial or project logic
which is mostly about efficiency
it's about expertise but in a very
technical sense
and it's also about innovation and also
disruption
so disrupting traditional old-fashioned
ways of doing things to create something
new
and again this goes at the cost of a
democratic logic
um but i also
uh
like to connect it to this uh work from
uh finzo and russell that you may know
uh which is called the innovation
delusion which is
about saying
they really show that it's also in our
point of time in certain societies where
innovation as a value is just
highly valued more highly valued than
other values and specifically the values
that are opposite to innovation which
would be
maintenance
maintenance work care work so anything
that that that works more on the
background to keep things going but
isn't disrupting it's actually enabling
society to run so think about
maintaining uh bridges roads but also of
course care work health care work these
are things that we value
a lot less in our societies and even we
sometimes make them very invisible
cleaning jobs are always done at night
or early in the morning so you don't see
it
and innovation that's something uh we
are we always make it very visible we
show it
um and there's also a gender dimension
to this um which i think is interesting
but we
i'm just curious to hear what you think
about it but there's definitely this
idea that there's something about
innovation which is
uh exploring discovering uh like these
thoughts of fronts here and breaking the
frontier and um
this is valued uh and it's also
stereotypically a bit of a
there are a bit of male values at least
stereotypically in our society whereas
care cleaning maintenance they are
stereotypically seen as female values
and therefore they're also
seen as less valuable
and finally but definitely not least
it really resonates with
afghanistan's idea of technologism
um so uh
sorry i just now realized that there are
more afghanis working on this than
just one
but uh yeah so so what we really saw is
that um
yeah the civil servants they are not
really led
by a problem when they're trying to do
these data projects not always sometimes
they are but oftentimes it's not the
problem that's the thing they just want
to do the data project and they try to
find any kind of problem
or even make one up
in order to be able to do the data
project so it's solution-led they first
have the solution they just need to find
a problem that they could
stick to it and that's of course uh yeah
leads to all kinds of problematic
aspects
so these are just
quite random thoughts and i'm just just
to hope that in the discussion you can
help us along in thinking about this
um
yeah so i think we're going to do future
outlooks yeah
um
so
i'll keep this brief because i'm not
sure so i i have um we have seen of
course over the years things changing as
well by doing these workshops for so
long uh and edis will have more to say
about it because she's been doing it for
even longer i've been
out of it for two years but i think
there is uh some hopeful messages so i
think we do see also in the public
debate about this but also within this
workshop that there is more and more
attention for
uh the democratic character of
ethical deliberation so there's more and
more criticism on having just a list of
values
uh and not really doing anything with it
not really doing anything deliberative
or democratic with these values
and i think that's a good thing we need
this kind of a shift
and i also think that we do get more
critical of this idea of innovation
being the only important value and we do
get more critical of techno solution is
thinking
um
yeah you may notice this is also just
what i hope
will be the direction that we are
heading but i think there is some reason
to be a bit optimistic there
but the third point which is not so
optimistic is that i think
specifically also when looking at uh
government institutions there's still a
large dependence on external parties and
it depends very much on the kind of
project that you're having what kind of
parties they are
but they they have quite a lot of power
when it comes to data projects
and also when it comes to public policy
uh as data projects and and this is
quite worrisome so there isn't there
still isn't a lot of uh
um
i i still think there isn't enough
initiative on public institutions
themselves to start developing uh the
expertise needed to really keep uh data
projects in the democratic system and
this of course also goes on a different
legal level when we look at big tech who
are also still increasingly uh getting
much more and more powerful within these
kinds of projects um i did some research
into corona apps on a european level and
of course this is a very clear example
where big tech had such a big say
in our ethical discussion about what
these technologies should look like
um so a bit of positive and a bit of
negative and now i hope edith will make
it even more positive than i did
well let's see
thanks lucia
um
so
i've really noticed a very big shift in
uh the sense of urgency surrounding
digital ethics so
when we started at it school um
interesting or getting interested into
data ethics which was around 2015.
really no one was talking about this we
even did a
small research into how many times the
forthcoming gdpr was mentioned in the
media
zero
zero times up up until like the the
the last two months it was actually uh
getting into effect in in 2018 so it
really shows the lack of awareness
surrounding these topics and the past
couple of years i've really seen a very
uh
very high rise in
the attention given to data ethics so
there are a lot of
new frameworks being released uh
um
guidance instruments guidelines um
codes of conduct
uh especially in the past year or so
like year one and a half years so
um i feel like that's that's a very
positive thing um
there's also an eu regulation uh
forthcoming um
about a.i so uh i'm just very curious to
see
what its final form will be
and it really changes
the the character of of doing data
ethics from uh
being up until now purely voluntarily
or mostly
um
to
being a bit more obligatory
so
i also feel like that's
that's a good thing because now doing
data ethics is largely dependent on
individuals within organization
organizations that have an
intrinsic motivation
in
talking about these subjects
sorry let me take a sip of water
so um
so the character of data ethics will
really shift uh in the future um
i think it will be very challenging to
to codify ethics because
of course ethics as opposed to law
is a very gray area which is not really
um
it's very hard to put it into print and
to to to codify it
so instead i feel like uh
demanding proof or documentation of an
elaborate ethical deliberation will be
more successful
so really
instead of demanding
um complete
adherence to the law
demanding elaborate documentation and
and showing that you have carefully
considered uh ethical considerations
within your your process
um
so uh
that's that's just my that's just my
opinion i'm i'm interested in hearing
your uh opinion uh in a minute
uh something else i
wanted to
talk about is that there is um
a new impact assessment which is the
fundamental rights and algorithms impact
assessment
which was created by the utrecht
university last year
i was one of the the co-developers of
this impact assessment which
i
think will play an important role
especially in the dutch context because
this uh impact assessment
um is one of the the options
when impact assessment are are made
obligatory by this eu regulation so it's
it's one of the options of
uh for for the dutch context at least so
it is available in dutch but it will be
available in english i hope
this month
uh
and i will definitely make sure to share
um
the link with uh jeff gainey
if you're interested
and this this is really also focused on
that uh um
facilitating
uh on on facilitating the um
documentation
of ethical deliberation so really
focused on on on creating that proof of
a careful decision-making process when
it comes to ethical aspects
so um
we would really like to hear
your opinion on this um
i've written down some you know starting
questions on on this slide but
just feel free to weigh in on on
whatever subject we have talked about
today
um
and um
[Music]
yeah let's let's open the floor uh up
for some uh some discussion i would
really love to hear some uh some
reactions of you
thank you so much uh roger and it is
fascinating insights uh so indeed let's
open up the floor for uh for questions
and discussion so
please feel free to use the raise your
hand uh function or send in uh into chat
but
preferably just uh
use the raise your hand function so that
we can have a
chat uh
in live format i i did see there was one
question
uh that came in in the chat earlier uh
from nishant
nishant would you like to ask a question
yourself
ah sorry i've missed that question
otherwise i i can i can read it so
nishant was saying i'm sorry if i missed
this but i have a question what kind of
infrastructure
were these projects using uh for example
compute for data processing storage
machine learning etc
so i'm guessing um the question is about
what cases were typically brought in for
for this dita instrument
um so this this really varied a great
deal so
sometimes a case was brought forth
surrounding an algorithm to predict
which citizens were likely to get into
debt for instance
this algorithm could be developed by an
external party
or could be developed in-house
another example is
you know
this is of course a like a risk
assessment model and these were
these were very prominent within the
cases that were brought forth but we
also encountered uh more more simple
data projects so it it it really
depended
uh completely every workshop was had a
different case being discussed
i hope that um
answers your uh your question
thanks and uh
next we have mandalay
yes
oh can you hear me yes
yeah okay sorry my computer's a little
funny
um yeah thanks so much for your talk it
was so interesting i was especially
interested in the the zeitgeist
narrative um i actually
i work in the ethics and philosophy
section at the university and and work
with engineers as well and i've heard
this a lot in in both the philosophy
section and the with the engineers as
well
and i was just wondering
you
how often you hear explicit concern
about funding
so we have to keep up because funding
that's how we'll keep
uh
funded or
um i guess i'm just wondering how how
money
plays a role in the zeitgeist narrative
and if it does explicitly um which i
think you sort of allude to in the
references you were talking about um
and yeah that's what i was wondering
yeah it's actually a good question i
think um so i feel like with the the
workshops we did with these local
government institutions
it doesn't explicitly seem like funding
was a very big aspect of it um it was
more like the um
so the political interests and the uh
the political image that could be
created with these data projects that
seemed like a big
a big influence but please either sir uh
say so if it was different i i also just
wanted to briefly mention that when i'm
because i'm currently also looking at
the influence of big tech and healthcare
and there it's very clearly uh present
so
when you ask medical researchers why
they want to collaborate with
big tech
it's they just say there is no public
funding for these kinds of projects
because usually these data projects are
long-term
research projects and they just say that
there's no public money so we have to uh
and and it's of course a
very worrying answer yeah so so
uh and i think it might be relevant here
as well but
much less explicitly
it is anything you want to add to that
or no i i completely agree so um uh
municipalities have a set budget for
things like um
social services
and when they deploy an algorithm or
some other type of data project within
the domain of social social services it
comes out of that budget
so
typically it's not something that's
really
being made explicit
funding within these types of ethical
deliberation but i i really think that's
due to the the structure of the the
municipal organization and and the
budgeting
yeah but but very interesting logic to
hear the the the very big difference
between what you've encountered within
healthcare
yeah
um catalan you have a question i
see yeah thank you very much also for uh
the interesting presentations and
documentation that you provided i'm
really
quite interested in what you do with
dita
and i see so publications coming up and
you have an interesting website on it as
well so what do you see as the next step
in in uh in this this platform what what
would you like to enhance over time
in dita you mean
yeah so i'm actually working on an
update right now
so we aim
to bring out an update every year
due to
technological developments
things we encounter during workshops
uh you know changing uh
changing laws and regulations
surrounding the topic
um
so that's that's something we we take
into consideration
um and for this
new impact assessment that we just
brought out yama
we um
we did that because there we felt a need
that there was uh for more focus on
human rights so this impact assessment
therefore
focuses heavily on human human rights
fundamental rights
and also it narrows the scope from data
projects in general to algorithms in in
particular
uh because the the need is is changing
the the use of algorithms that could
potentially breach human rights is
increasing
so that's that's a shift we've
encountered
so is this like an ongoing collaboration
between radbout and utrecht
[Laughter]
um well no it's not so uh the basis for
this i mean we're still uh thinking of
ideas together and collaborating but
it's kind of uh um what i'm doing at the
about is kind of a different project
so it's just uh i used to work at
utrecht and now i work at that but of
course the the the issues are very
similar sometimes and there's very
interesting so we
we collaborate and exchange ideas yeah
but it's not a uh the project that at
the dot about is is a different project
yeah
on a similar tour
okay thank you
but of course we hope it's an ongoing
collaboration
yes uh so do we ah no yes please go
ahead
hi uh thanks um
for the fascinating insights
um
i was going to ask you um you're talking
now with with engineers all of us i
think
most of us have a background engineering
not all of us but i think all of us are
in this ai tech community really to
to build bridges
but also to see how we can transform
the culture and practice of engineering
so i was wondering based on your
insights because there was like
i think it's a really great
uh granular insight into cult like
engineering culture or more particularly
i think
cultures of
um
techno solutionism that are mostly i
think
inhabited or also
persist
through uh you know engineers uh taking
on these roles in these spaces
so i was wondering if
if you have any advice or any ideas
about
what we like what we can take away or
what we could do
based on on these insights of course i
have my own but i was curious to see if
you have any any any ideas for how we
could train and engage
with the next generation of engineers
computer scientists
to uh yeah to
steer away from from the from this
culture and work towards other practices
it's such a
good and also big question yeah no no
it's uh
so
for me but that's also because that as
you also saw and which is also ingrained
in this guy's narrative one of the
really important things is that we need
to
yeah just uh develop systems of
recognizing when this narrative is there
or when we are thinking more in a
techno-solutionist way than in an actual
um
yeah a
constructive ethical deliberation and i
feel like
yeah we're getting i do think we're
getting better at that but we still need
a bit of um
yeah just kind of kind of recognizing it
so that we see like oh
it seems like we really want
this solution and uh we're not really
thinking carefully about whether
this problem that we're trying to stick
to it is really a problem is really
something that needs to be solved in
this way or maybe a much more complex
problem or it may be that we're creating
more problems as we're trying to fit
this solution to it um
so we need
yeah systems of thinking about this and
i think actually so uh this is also
the work that the data score is doing is
also always
uh re-evaluating
uh the the kind of
pretty practical tools that are there
but they are tools to open up the
liberation so you always have to think
about how do we keep the process
deliberative how do we keep thinking
about the democratic values involved in
there and i feel like that's really the
key because if you
this is the thing with ethics of course
it changes and that's its essence it
should be allowed to change and any kind
of thing that
tries to pin it down too hard or
actually tries to preclude any kind of
ethical deliberation is dangerous so
these are for me like the fundamental
uh conditions that we need for ethics
that
they're mostly democratic values because
those think about how to make them
transparent how to allow ethics to also
keep changing
and i i think engineers uh are
getting better and better at thinking
about this but they
they have like a really big
responsibility because they they know
the details right and this is i mean
this is why what you're doing of course
is great because
in order to think about the ethics and
this is why i also showed you like how
much we noticed how much of the
difference it makes when you have the
people who are good at thinking about
public value together with the people
who know how the technology works that's
really the only way to think about it
ethically if you miss one of the two
it's not really working
but i'm also interested in your uh
your ideas
definitely and to to add something super
concrete uh
uh
to what you were saying lucia so to give
one example of how we deal with this is
we we have a master program applied data
science
uh and we have now started to integrate
an ethics weekly ethics colloquium
within that uh
program so it's a mandatory colloquium
for all of the data students
and every week we either invite someone
from the field to talk about
like a data scientist to talk about how
he or she
deals with data ethics
this week i am giving a session there
with dita so with the data ethics
decision aid
to
you know really teach the the students
to have these deliberational processes
surrounding ethics yeah but what is your
view on this whole
well first thanks of all for the i think
there's really tangible things so you're
talking a lot about systems of thinking
and reflecting on the problem
the stakes the different different
perspectives
staying open to new forms of ethical
deliberation so like also reflecting on
the practices of the liberation and the
tools that you're using
um and then i heard um
a few more things but hopefully
we're all taking notes so i think first
of all thank you for that
um well i think you know
indeed we are
at least here in delft trying to create
um
more and more community and more and
more ways for
other researchers and students to
to engage but
i think we can learn a lot from from
what you're doing
um and
yeah i think
we i think we're ready so personally i
chose to come to delft and to work in
the technology policy and management
faculty because it's
kind of
just this rich
collection collection of
different disciplines
that's also then translated to the
curriculum
and so i think within delve we can we
can learn from that we can also learn
from our our colleagues in in industrial
design engineering that have a similar
kind of makeup
um so there's a lot of i think quick
wins and that and then also i think
inspiration we can draw from the kinds
of programs and tools that you're
developing
so those are my immediate kind of more
pragmatic thoughts
uh
yeah
not much to add i think it's excellent
and
well i think one one key thing that
we we still lack and this is where i
spent some of my research is trying to
work towards ways to imagine new ways of
designing
data-driven
functionality algorithmic functionality
so i think if you if you stay stuck in
the on the one and the very practical
nuts and bolts and on the other end the
more i like kind of ambiguous values
like there's still like a big gap there
and like how do you fill that with like
ways of thinking how to bridge that also
in a material way so some of the work
we're doing is
like what are the kind of
typical socio-technical dimensions that
keep coming back when you look at a
system once it's integrated in a in a in
a context
so what are the kinds of things you have
to take into account when you're
designing these systems both
materially kind of but also in terms of
how you how you manage them but that's
really ongoing work so it would be great
actually to kind of bring that
perspective together with what you're
doing and see how it could
kind of complement and um
and feed on each other
our cat also wants to be part of the
conversation
so
uh
come here so
um
that something that i observe is that
you know often how we frame projects
that
we we start from saying
we want to use algorithms or ai to solve
societal challenge x
and that's kind of how the project
starts
and this is what i often find
problematic because
for example
what you talked about today that is
often on my mind
and i find but that leaves very little
wiggle room to even raise the the the
question but hang on uh so the societal
challenge we want to solve that great
uh but
but if we if we enter the project with
the framing that
the solution
involves algorithms and ai how can we
even question that maybe the solution
does not involve algorithms or ni or ai
or maybe
it might involve them but in a very
different
way and narrative from
you know what is being raised at the
beginning well
what do you think about that
yeah i think it's so i think you're
completely right and it's also one of
the things that that interests me about
these kinds of projects it's indeed the
yes yeah so like
it's
you can sense that it's more solution
driven than problem driven and then
also the problems that they're trying to
address are really
big and complex and need
care
in order to think about them and to
develop uh solutions about them but it
but this is indeed something that we've
noticed and i'm i'm still noticing also
with different projects that it's it's
um
yeah it it it's technology led so so
this whole
there's still just this this
way that we
we worship a new technology in
progress and i feel that that's really
connected to this because then you
you have this technical solution and
that's what you're actually starting
from in your thinking process
and i think you're right so it precludes
a good ethical deliberation and i also
think that there's something interesting
in the
the this is also why i emphasized haste
because i think that's also there in
this way of thinking it's always about
doing things very fast
whereas i think inherently in ethical
deliberation and democratic processes in
general
they need to be slow they need to be
slow right because you can only do it by
thinking about it with a lot of people
letting it pass through a lot of
institutions having all these moments
where it's slowed down and refute again
so so there's an inherent tension there
that that yeah
um we need to be wary of or maybe we
need to you know this is also why i like
the
idea of maintenance work because these
finso and russell they're really
thinking about okay so let's start
worshiping maintenance instead of
innovation and you could also do this
with slowness like that's worship
very slow bureaucratic processes let's
enjoy them
and that's not that's not something
we've we have scripts for how to do that
now but if we develop those that could
help yeah
i don't know if that
kind of resonates but yeah yeah yeah and
and also
no the thanks lodge and and uh and and
also to echo some of the things that you
were not talking about with rule uh yeah
personally uh i also think that a lot of
um
a lot of the things that we need to do
on this frontline the educate in the
education realm so so
uh
because looking back for example at my
journey i see that
there's a lot of
socio-technical gaps and systemic gaps
in terms of you know how i was educated
from my technical background to think
about problems that is very like well
you solve uh
that in today's challenges basically
that i think there's so much we can do
in the way we train the future
generations of specialists that they
think systemically and they realize it's
a puzzle that you solve together with
other disciplines
and i think so so indeed like rule was
saying i think we can really learn from
each other about the things we're trying
and experimenting and education in our
different institutions
yeah i agree
yeah
so unfortunately we ran out of time so i
think we could
continue talking about this for much
longer
i i want to
launch and it is i want to thank you
both very very much for coming today and
sharing your insights
and i hope we can continue this
conversation another time
and of course will be great if uh
uh people uh uh people would like to
reach out to each other and talk more uh
offline
yeah great sounds good
yeah thanks so much for the invite and
then
maybe hopefully speak to you again all
the time i will talk again at some point
great
definitely thank you thank you thank you
very much for joining today take care
bye bye bye
you