199. How close are we to creating Artificial General Intelligence

okay hello everybody
um welcome to the
199th edition of um
yeah i safety denmark reading group
uh i'm presenting it tonight i'm chris
cooper
and we're talking about the ideas of
the physicist david deutsch on
artificial intelligence
david deutsch is uh a visiting professor
of physics
at um oxford university
um he was a founding member of the
center for quantum computation
and that's what he is mostly known for
that he as a pioneer of computation
um his citation for being elected a
fellow of the royal society which i'm
showing here
explains that he laid the foundations of
ethereum computation and he's made a lot
of
contributions subsequently
he is known for being an advocate of the
multiverse interpretation of quantum
mechanics
he champions uh cal popper the
philosopher
of science and he applies
poppa's ideas very strongly in
as we'll see throughout this paper and
uh david deutsche is also known for
developing um
a personal uh
theory of physics or um organizing
principle of physics called constructive
theory
which apparently seeks to express all
fundamental scientific theories in terms
of other equality between
possible and impossible physical
transformations
and that is all i know about
constructive theory
this discussion is centered on a paper
um
[Music]
written very much at the popular level
uh it's called creative blocks it
appeared in the magazine
eon i've called it how close are we to
creating artificial intelligence because
let's use a better
title and it actually appears in the url
of
this paper um the paper argues that
um we've made virtually no progress in
developing
agi to date but nevertheless we
shouldn't be downhearted because we may
very well be very close to doing so
now um david deutsch's key claim
is that agi
general artificial intelligence must be
possible
he reckons that he is the one who has
proved that
um he says that everything that the laws
of physics require a physical object to
do
any physical system including in
particular the human brain
can in principle be emulated in
arbitrarily fine detail
by some program on a general purpose
computer
provided is given enough time and memory
he says that the first people to guess
this and to grapple with its
ramifications were babbage and
lovelace back in the mid 19th century
um in the 20th century cheering
he says fully understood universe
universality
but um and you may would have thought
that he'd actually
um prove universality but though which
says that
it actually remained a guess until the
1980s when
david deutsch himself proved it using
the quantum theory of computation
now turing concluded that a computer
program whose repertoire included all
the distinctive attributes of the human
brain
including feelings free will
consciousness and all
could be written um i haven't had time
to go back and look at turing's um
classic paper of 1950 which we discussed
in this group and i'm not quite sure
whether he actually said this or whether
he was proposing the famous turing test
as a way of sidestepping
these very difficult questions just
giving an operational test as it were to
replace them
so i should be going back and looking at
that at some point but
i'm sure actually deutsch probably knows
a lot more about what tiering thought
than i did
david says that this astounding claim of
turings
split the intellectual world into two
camps
one insisting that agi was nonetheless
impossible
the other one insisting that it was not
only possible but
imminent and
deutsch says that both were mistaken
the first camp thought that ai is
impossible because of the
of the unique properties of the human
mind um
and that tend to you know possibly be
supernatural qualities or ineffable
qualities to do with morality and
aesthetics and religious sense and
subjective feeling and stuff
this camp failed to understand
universality of computing
in other words that um
the human brain is a physical object and
therefore all its workings
can be simulated on a computer
the second camp thought that ai is
possible and imminent
they fail to realize that the
functionality of the human brain
stroke mind is different from
all other computer functionalities
deutsch says that the core functionality
required for ai
is creativity specifically
creativity in producing new explanations
now the concept of explanation
is central to deutsche's thinking
it has to be contrasted with the mere
discovery of correlations
um or even
the mere production of successful
predictions i say the mere production
obviously that's a big thing but
um even that isn't quite the same as
explaining
it's pretty much synonymous with
successful theory
whatever that means it and that of
course does include successful
predictions
anyway um explanation is very much a key
term
for deutsche uh deutsch uh contrasts
programming of the computer to do
something very simple like converting
temperatures from centigrade to
fahrenheit
um he contrasts that with programming
and agi to address
a problem in physics such as the nature
of diet matter
the temperature conversion could be done
with a lookup table or it could be done
by means of an explicit calculation
and he seems to regard
current ai achievements to date as
similar in kind to this
sorry i'm throwing up extraneous
excuse me
onwards so with regard to
um creating a new physical theory
he says suppose you were somehow to give
the agis program as a list of
explanations of dark matter that would
be acceptable
outputs of the program if the program
did output one of those explanations
later none of those explanations would
be new
you would already have created them
yourself in order to write the
specification
okay i don't think anybody would argue
that that
wouldn't be too impressive
he says we need an agi algorithm with
the right functionality
in other words able to create theories
that does not involve first making new
discoveries in physics and hiding them
in the program he wanted to do something
that we can't do
in advance
researchers have discussed tests of agi
but these give no clue as to how the agi
is to be implemented the turing test for
example
gives no clue as to how it can be passed
and he mentions in this context uh
evolutionary algorithms there's one
technique that
cannot succeed um i don't know how the
unfortunately
person who's um reading this article
um it's supposed to cope with the idea
of evolutionary algorithms because he
doesn't
explain much but um
okay obviously as i'm sure everybody
here knows
uh evolutionary algorithms involve
uh producing some trial agi programs as
test candidates you judge their success
you select the most successful
you make copies of the most successful
ones with slight
random variations you then test those
select the most successful of them
and you repeat the process for many
generations and you accumulate
lots of small improvements to get an
improved algorithm and that's
of course a technique which is used to
develop
um improved
algorithms now narrow ai algorithms
deutsch says that the turing test cannot
itself be automated
without first knowing how to write an
agi program
since the quote judges of a program need
to have the target ability themselves
in other words
[Music]
here the judges are the parts of the
overall automated procedure that assess
the success
of the candidate programs they are the
equivalents of human judges in an
ordinary turing test
so he's saying a turing test might work
because human judges
know how to judge human behavior um
an automated version would also need to
be able to judge that and you couldn't
do that without
essentially having written your agi
program in advance
you've got to have the ability that
you're supposedly
testing
he says that agi cannot possibly be
defined purely behaviorally
because in the classic brain and of that
thought experiment
the brain when temporarily disconnected
from its input and output channels
is thinking feeling creating
explanations it has all the cognitive
attributes of an agi
so the relevant attributes of an agi do
not consist only of the relationships
between inputs
and outputs
so you can't judge an agi by its
behavior because you could cut off all
its
means of behavior and it would still be
a functioning agi
the upshot or this is that unlike any
functionality that has ever been
programmed to date
this one can be achieved neither by a
specification
nor by a test of the outputs
that's rather dense conclusion
um
and very arguable i'm sure we have a lot
to say about it
what is needed is nothing less than a
breakthrough in philosophy
a new epistemological theory theory of
knowledge
that explains how brains create
explanatory knowledge
and this theory will have to define in
principle
without ever running them as programs
which algorithms possess that
functionality and which do not
so in some way we've got to be able to
create these algorithms
and know that they can do the job
in advance of running this program and
getting them to do that job
and as i say as you will gather it from
my tone it's not clear to me that this
is so
but uh i'm sure people here will have
lots of opinions about that
now the prevailing misconception in ai
he says
in our research is that by assuming that
the future will be like the past
one can quote derive or extrapolate or
generalize all these things in snare
quotes
that one can derive these theories from
repeated experiences by an alleged
process called
induction but that is impossible
his inspiration throughout this work is
karl popper as i mentioned
um right at the beginning and popper
stressed the creativity of scientific
theorizing and the impossibility of any
simple method of extrapolation of the
future from the past
deutsch gives the example of the
calendar um the very simple obvious one
but actually i think it's
um extremely uh
relevant
he says that i remember observing on
thousands of consecutive occasions that
on calendars the first two digits of the
year were 19.
i never observed a single exception
until one day they started being 20.
i wasn't surprised
not only was i not surprised i fully
expected
that there would be an interval of 17
000 years
until 19 appears
in that position again because obviously
that will be the year 19 000
new year's day on 19 nineteen thousand
seventeen thousand years ahead of the
year two thousand
so that's a period that neither any
other human being had previously
experienced even once
so that expectation is not based on
in any simple way on experience
and i'm just adding the example because
i like it the experience of turkeys who
are fed by the farmer
for 364 days in a row and lacking an
adequate theory of the farmer's behavior
the turkeys uh being good inductivists
assume that the future will be like the
past
and that surprised on the 365th day he
comes with an
axe
back to deutsch he says it's simply not
true that knowledge comes from
extrapolating
repeated observations nor is it true
that the future is like the past
in any sense that one could detect in
advance
without already knowing the explanation
without an explanation any continuation
of any sequence
constitutes quote the same thing
happening again
unquote under some explanation
so you can say the calendar is always
doing the same thing over and over but
according to a rather
complicated rule
he says in regard to how the agi problem
is perceived
this casts thinking as a process of
predicting that future patterns of
sensory experience will be like past
ones
that looks like extrapolation which
computers already do all the time
once they're given a theory of what
causes the data all right i won't do it
on that
shoot on
and he says something which i find
extremely interesting
in reality only a tiny component of
thinking is about prediction at all
let alone prediction of our sensory
experiences we think about the world not
just the physical world but also worlds
of extractions
such as right and wrong beauty and
ugliness
the infinite and the infinitesimal
causation
fiction fears and aspirations and about
thinking itself
and to me that that's sort of very
pregnant
let's i'll just go back
um i just seem to profoundly important
to me and just as a reminder of the
richness of thought and it helps to mark
off the
the field of general ai this is this is
what
a general ai would have to be like a
concern itself with and
it's very strongly strongly marked off
from that of the narrow ai that we're
familiar with
now he has a he has a jab at bayesianism
not against the reverend thomas himself
and not against basic statistics but
just again
making it you know the bill and end all
of uh
ai research uh he says that the doctrine
of bayesianism assumes that
minds work by assigning probabilities to
their ideas
and modifying those probabilities in the
light of experience as a way of choosing
how to act
and that behaviorist input output model
is appropriate for most computer
programming other than aji
but hopeless for agi
and he says it's ironic that um
mainstream psychology has largely
renounced
behaviorism the simple input output
model
whereas ai research is
wedded to it i'm going to be using a
pointer possibly um from
a lot from now on because um
my uh i ran out of time to do the
animations
okay jeopardy the um
famously um the i it's the ibm it is ibm
isn't it um
watson program um
[Music]
won victories on the ustv
quiz show called jeopardy jeopardy with
an exclamation mark at the end
[Music]
um
and it was very impressive and it was
hailed as you know i robot overlords are
on the way
um jeopardy i gather
is pretty much a general knowledge quiz
but you have to also do a bit of
ingenious connecting of um
the question you're asked to the answer
that's wanted
um he says playing jeopardy like every
one of the computational functionalities
at which we rightly marvel today is
firmly among the functionalities that
can be specified in the standard
behaviorist way that i described above
no jeopardy answer will ever be
published
in a journal of new discoveries so this
is nothing like
scientific discovery
there's now a long stretch of
misconceptions which he
attacks another hopeless approach to agi
is to start from existing knowledge of
how to program specific tasks
such as playing chess performing
statistical analysis or searching
databases
and to try to improve those programs in
the hope this will somehow generate
aji as a side effect has happened to
skynet in the terminator films
skynet was a global defense network
which just got so big that one day it
just woke up
became self-aware and then started
carrying out his evil plans for the
human race
and he says that a similar misconception
the whole misconception that
increasing quantity will turn into a
change in quality
um this kind of misconception is
informs these ideas
that agi is merely an emerging property
of complexity
or that increased computer power will
bring forth agi
he says that a similar misconception is
that the unique abilities of the brain
are due to its massive parallelism
ought to its specific neuronal
architecture
and he says these violate
computational universality in other
words
anything that can be done by means of
these special properties mentioned here
that can be achieved with them can also
be achieved but
by any um
it can be simulated by any computer any
turing machine i suppose i
i have to speak carefully because i'm
not questioning what's a turing machine
or it's a quantum turing machine or what
yeah but they can be simulated
by any computer by other computers
but this argument seems suspect to me
just because um
i can imagine somebody producing a
practical computer design
um and it being criticized on the
grounds that uh
you don't need this practical computer
design because it could be
simulated by a turing machine um
so i'm not quite sure of the validity of
this
anyway he ends with um a nice
uh he doesn't end he goes on quite a bit
but uh he says at this point that
expecting to create an agi without first
understanding in detail how it works
it's like expecting sky's great words to
learn to fly if we go
tall enough
um a frequent phrase often one that's
attracted me
until i read deutsche as uh anything we
don't yet know how to program
is called human intelligence we just
apply the term
human intelligence to the gaps between
what we can program but what he is
saying
is on the contrary that when we have
when we achieve
ai its intelligence will like human
intelligence be different in kind
from other computer functionalities
it won't just see that there is that
we've programmed
now i won't finish that thought i'll
move on
i apologize once again giving you rather
crowded slides they were supposed to
build up in a nice smooth way but as i
say i ran out of time
he wants to insist that the mind is not
metaphorically a computer it is
literally a computer
he quotes uh john cerrell
john cell is is he of course of the
chinese room argument and we see him
here
in his chinese room he's in his room
getting inputs
in chinese um looking at his rulebook
and um shoving out
outputs in chinese perfect chinese
even though he doesn't understand the
word of chinese himself
however this is strictly irrelevant and
it's just there because i like the
i like the pick um soil here
is making the point that um the mind has
often been modeled
on the technology of the day whether it
was clockwork
or hydraulics or
steam engines or the electric telegraph
and he says that we nowadays we are
modeling a picture of the mind on
computers because computers are the
and the latest and coolest technology
that we know about
but uh deutsche says
i'm sorry joyce says
no it's not a metaphor the university
universality computation follows
from the knowledge of physics his
particular contribution of the 1980s
um so there
here's yet another um well another claim
that he combats
roger penrose the mathematician is one
of those who has suggested that the
brain uses quantum computation or
even something beyond that um
and uh that this explains the failure to
create
agi on existing computers
but deutsch who want to know since he
founded quantum computing or as
a founder of quantum computing says
along with most
researchers in his field that um they
disagree
this is a plausible source of the human
brain's unique functionality
and yet another misconception is that um
existing software
is already intelligent in a sort of
primitive way
as are um animals in a primitive way
and that we because of our
anthropocentric manatee
we don't recognize that it's continuity
with our own intelligence
and
as part of that uh another misconception
is put great weight on self-awareness
and deutsche says self-awareness in the
behavioral sense
for example to pass the mirror test
of being able to use a mirror to infer
effects about itself you know
chimps and apes and apes and monkeys can
look in
look in mirrors and notice that um
experimenters have put something on
their forehead or something like that
and they'll immediately brush it off
um this is a fairly useless ability as
well as a trivial one he says
at least it is as far as robots and ai
systems you can say
it's true that agis will be
capable of self-awareness but that is
because
they will be general intelligences
capable of awareness of every kind of
deep and subtle thing
including their own cells now
um self-awareness is closely
linked with consciousness
i hope everybody's watching the the red
dot which i will go around to try and
tell you
which bit of the screen i'm looking at
um
consciousness um
has a huge range of meanings at one end
of the scale there's the physical
problem with the nature of subjective
sensations or qualia
which is intimately connected with the
problem of agi so he does
do i just attach great importance to
this
i think i said last week in our
discussion but um
i uh i can't remember that i said i was
dreading having to talk about qualia or
whether
i was glad i wouldn't have to as it
turns out he says nothing about it
really except that it is
the problem of subjective sensations are
intimately connected with the problem of
agi
but he has nothing more to say about it
really
still talking about the the concept of
consciousness he says that
um at the other end of the spectrum
consciousness is merely that which we
lose when we put under general
anesthetic
somewhere disconnected from
sensations for a while and animals
certainly have that
um he attributes the great importance to
it
now a key absolutely key assertion of
his
agis will be people
um
that they are people has been implicit
in their concept from the outset
if there were a program that lacked even
a single cognitive ability that is
characteristic of people
then by definition it would not qualify
as an agi
and you should remember that deutsche's
very wide conception of what is a
cognitive ability um
earlier in this paper he described exp
experiencing boredom as a cognitive task
now um he makes
i i've just imported
a sentence defining his idea of
personhood from his book
the beginning of infinity strongly
recommended
um but i've brought in this um
sentence from there where he says people
whom i shall henceforward define as
entities that create
explanatory knowledge
and in this article that we're
discussing here that definition crops up
and is asserted as a fact
the fact that the ability to create new
explanations
is the unique morally and intellectually
significant functionality of people
meaning both humans and agis and that
they achieve this functional
this functionality by conjecture and
criticism
this changes everything he
um as i said he regards explanation as
key
and he carries it to the point of
regarding our ability
to create good explanations
um that increase our mastery over
nature and over ourselves um
he regards as as definitive of
personhood
and in the book beginning of infinity he
connects this with
the technological power it gives us
essentially unlimited technological
power whereas
all other animals have
evolutionary niches which they're more
or less confined
we are virtually unconfined because
of our um
unlimited you know technological power
i haven't digested this argument so i
can't really say much more about it
but this is key
so he then starts thinking about the
implications
of the fact that an agi
will be a person it is not the computer
but the running program that is a person
once an agi program is running in a
computer to deprive it of that computer
would be murder or at least
false imprisonment if you were putting
it in some other hardware
or it would be slavery
this when
agi arrives and we have the ability to
instantiate each program
with multiple copies deep philosophical
problems are going to
arise i think
our copies of the program while they are
still executing identical steps
before they have become differentiated
due to
random choices or different experiences
will they be the same person or many
different people
do they get one vote or many if you
delete one of them
is that murder or a minor assault
that is some rogue programmer perhaps
illegally creates billions of different
agi people
either on one computer or remini what
happens
next
to treat agis like any other computer
programs would constitute
brainwashing slavery and tyranny
and cruelty to children too for
programming and already running
agi unlike all other programming
constitutes education
in fact he says we have to forget almost
all existing connotations of the word
programming
to ignore the rights and personhood of
agis would not only be the epitome of
evil
but also a recipe for disaster creative
beings cannot be enslaved forever
he goes on uh in the same vein as some
hope to learn how we can rig their
programming
to make them constitutionally unable to
harm humans
as in isaac asimos laws of robotics
or we want to try to prevent them
somehow from acquiring the theory that
the universe should be converted into
paper clips as imagined by nick bostrom
once again i can imagine a reader of eon
who
was new to the subject being completely
baffled
by that sentence but of course we are
all familiar with the unintended
consequences that bostrom imagines of um
trying to try to create a super
intelligent paperclip
maker he says
deutsch says none of these are the real
problem
little aside from me uh my suspicions
are always aroused when someone says
that
because it always means they are about
to divert attention to another problem
he says it has always been the case that
a single exceptionally creative person
can be thousands of times as productive
good or ill as most people
that therefore will be true of an agi
these phenomena have nothing to do with
agis the battle between good and evil
ideas
is as old as our species and will
continue
regardless of the hardware on which it
is running
okay
possibly complacent
so he says um how should society be
organized
so as to promote that improvement
i'm sorry i didn't seem to have
mentioned what improvement that was i
didn't think that was on the previous
no slide sorry about that the
improvements
um which means the improvement of
society in general or a society
which incorporates energy highs anyway
the slogan enslave all intelligence
would be a catastrophically wrong answer
enslave one intelligence that doesn't
look like us
would not be much better
just because it's made of steel and
silicon rather than
carbon-based wet wear
it's not a good reason for
discriminating against that
form of intelligence
he says learning must be something that
newly created intelligences do
and control for themselves
and um he does indeed have um
corresponding ideas about the education
of children human children
uh which
he believes can be done without any of
our usual coercion
and dogmatism
so
he says the whole problem of developing
agis is a matter of philosophy
not computer science or neurophysiology
and the philosophical progress that is
essential to their future integration
is also a prerequisite for developing
them in the first place
without papyrian epistemology this
epistemology of
creative conjecture
followed by criticism but not
not just bayesian induction or anything
like that
one cannot even be without such an
epistemology one cannot even begin to
guess
what detailed functionality must be
achieved to make an agi
and i'm making another grumpy comment
down here that unfortunately
preparing epistemology well certainly
necessary i think i
i i buy into that certainly but it's
clearly not sufficient
because david deutsch is clearly not
actually
giving us this
this hoped for philosophical advance
he says thinking of an agi as a machine
for translating experiences rewards and
punishments into ideas or worse
just into behaviors is futile
because it is rooted in an archaic and
widely mistaken world
view that's the false philosophy
of science that he's talking about if
one works towards programs whose quote
thinking
unquote is constitutionally incapable of
violating predetermined
constraints when it's trying to engineer
away the defining attribute
of an intelligent being of a person
namely creativity
but although we need this big
philosophical step
he says the information how to achieve
it must be encoded
in the relatively tiny number of
differences between the dna of humans
and that of chimpanzees
because that must be that which accounts
for the difference in abilities between
us and chimpanzees
so in one respect i namely deutsch can
agree with the agi
is imminent camp it is plausible that
just a single
idea stands between us and the
breakthrough
that's the end of his
article i'll just give my some responses
here quickly um
i must say that after i'm reading him
the idea of creating a super intelligent
slave
devoted to making paper clips first
seems like a crime it's always
it does seem like a contradiction in
terms
um all these discussions of super
intelligence which is
really literally our slave
um deutsche seems to be putting his
finger here a genuine dichotomy between
the merely skilled activities which are
narrow
ai is creating nowadays
and genuinely creative activities
but i think he's i mean uh i think he
proves too much we're in the sense that
he
too quickly gets rather complacent um
what we are creating now
uh with machine learning uh it's going
to lead to an accumulation of very
capable algorithms
which can still be very dangerous and
well transforming in
in all the ways we're familiar with and
um
we don't necessarily become safer even
if
we can convince ourselves that narrow ai
will never become
general ai
um i'd like to know more about um
existing attempts at theory creating ais
i mean i know
back in the golden age of ai there there
were all sorts of attempts at fear
improving theory creating
i'd like to know um how they're doing at
the moment
and i um certainly don't understand the
full depth of his remarks
when he said um
that we have to be able to specify aja
functionality ahead of actually
implementing it um i i don't understand
the
full depth of all that is it impossible
the sort of things that lead to gpt3
today could not lead to
a genuine agi which we we'd know when we
saw it even though
we wouldn't have been able to specify
detail how it happened
finally um granted that a creative
entity must not and cannot be enslaved
doesn't that suggest that we shouldn't
bring them into existence
i'm sure he believes i i haven't um
didn't have time to find a quote
to this effect but i don't doubt
that he believes that ai not only can be
but should be brought into existence
he generally thinks that life and
intelligence should be spread everywhere
so i'm sure that he would think that and
so he
really does want to summon up the genie
from the lamp
um and this seems
it seems a bit rash to me in other words
he's not just um
happy if he hears um
intelligent aliens in distant space
announcing that they're going to be
arriving in 50 years
and that we should get ready he not only
welcomes that he would
uh he'd be impatient and he'd want to go
out and fetch them
that's my impression okay
and that is uh all that
i have to say on that paper thank you


226. John Fox on Is AI Safety a Progressive Research Programme

hello and welcome to session 226
in the ai safety.com reading group
tonight john fox will be giving a
presentation
john fox is a interdisciplinary
researcher
from oxford university please go ahead
john
sarah and thanks very much um hello
everybody
um appreciate the opportunity to talk
about this
um i'm a bit of an outsider uh
i would self-identify for most of my
career as a cognitive scientist
and i've been attending these events
trying to understand the ai safety
program
and how it relates to my perspective on
ai
and cognitive science generally
um just the starting point
i'd like to choose is a book that
subrata das
and i wrote about 20 odd years ago which
was based on what now would record we
would call
a traditional ai and applied in medicine
and for us medicine was a a very
important challenge domain
because it raised huge numbers of issues
and questions
of many kinds as well as just practical
ones of what's often called narrow ai
some years later i was surprised to kind
of hear from
nick bostrom who sent me a copy of his
book um
and not long after that was invited to
spend a few days at
miri um
i think because they thought i was a
rather strange fellow who didn't some
uh uh kind of see
ai safety the same way which was sort of
true but what i've
increasingly been committed to since
then is to
understand the ai safety program
um and um and and how it fits
with the kind of things we've just been
talking about
pseudoscience and progressive science
and so on
um i think you probably all know
that you know ai has always been
multidisciplinary i'm a
multidisciplinary scientist because i do
cognitive science
um you know it's from the most simple
i shouldn't call it that but the most
familiar
kind of everyday psychology to empirical
experimental scientific psychology
these days more and more influenced by
neuroscience
but also a kind of evolutionary and
ecological dimension
and of course the philosophers have
always had a very strong
view of this um who they overlap
with the the kind of formal rational
attempts to kind of understand mind and
intelligence and then kind of round at
the bottom i'm not saying this is a
complete
picture but just to kind of scope this
out
that ai of course has a strong design
and engineering tradition as well
um as a scientist my own personal
interest not so much about it
today but is how do these programs
converge are they converging
are we understanding ai and indeed
natural intelligence
in a way that we is is really
progressing and extending
so i'm going to try to use that lens to
talk a bit about
ai safety well again being very careful
i'm
keen to try to
support a conversation between
the communities not not to sort of say
this is a correct way of approaching
things
so um we mentioned papa we mentioned um
um kuhn um the other
of the three horsemen is lakatosh
who um i just found a little quote it's
not from him
but essentially a research program
he views as a sequence of theories
within the domain of scientific inquiry
where each successor theory and it's
surely been oversimplifying um
is held to mark some sort of advance
over its predecessor
that's kind of my starting point i'm not
a philosopher of science or any other
kind of loss but
so um i'd say don't think i have a
uh i don't want to be taken as this as
authoritative
um there's many programs in cognitive
science philosophers have been
particularly
active in um which are
i would regard as informal i'll try to
be clear about
sort of things i mean by formula in a
moment um but
trying to expose the assumptions we have
about
human minds um and non-human minds
um including the hard problem that david
chalmers and many others talk about of
consciousness and subjectivity
which is some way outside
our current articulation whether in
philosophy or any other branch of
cognitive science um
i've always thought and this
preparedness talk has kind of tried made
me try to
be clear about this that cognitive
science aims to be progressive that it's
grounded in the real world
it is incremental in in the lack of
sense perhaps you could call it
ideological but
my interest has always been in
benevolent ai
though i worry increasingly about
malignant ais
but the thing that that has excited me
as a scientist for
almost 50 years is that um trying to
seek
a theory that is is convergent or not
necessarily only one theory
and and what i've tried to kind of
understand
in attending these events um is whether
the ai safety program has
um analogous aspirations
and what i've heard both here and many
other places is
the importance that's attached to
whether the ai is
in some sense humanly aligned it's
humanly acceptable transparent
predictable
and controllable um i i don't know
whether the
the community would regard these as as
as key
themes or a complete set of themes
but that's what i've understood so far
um and a lot of the discussion is
increasingly over the last
few years uh debate perhaps
is being expressed through a series of
books and this is just a few of the ones
i've read
recently um from stuart russell's
book to gary marcus and
ernest the book the book of rebooting ai
and mike waldridge's book on road
conscious machines
most recently brian cantrell smith and
they're all kind of addressing
the ai safety program either as a
computer scientists which are
stuart and and and mike woodridge
um um
and ernest davis with gary marcus
and brian cantrell smith
with the strongest psychological
framework
and that's a very active and ongoing um
set of publications which i'm sure
there's going to be more of
i said my particular interest is in
whether the cognitive sciences are
converging and i like to
view these sciences in terms of three
rough groupings
the experimental sciences observation
um interpretation hypothesis
and test but also more
that's typically psychology and
neuroscience but rational
sort of foundations the mathematicians
of many
stripes logicians former formerly minded
computer scientists
and people trying to build kind of
apply that the the theories that come
out of those
perspectives into in practical
applications
um and again that is a series of books
that i'm familiar with over a longer
period from
uh which is about which all in different
ways trying to say how can we bring
these three methodologies together
um in some sort of unified framework
um alan newell's book of mule and simon
fame
was published in 1990 um
uh had a particular very computational
but psychologically well informed
uh attempt to to bring um the ideas
together at the time
because there's one book not i haven't
seen anything like it since but paul
cohen on on
trying to buy experimental methods to ai
ernie davis as we um heard about
a couple weeks ago very much more formal
trying to understand common sense
reasoning common sense knowledge
and and and also work
huge amounts of work in the uh
neuropsychology and neuroscience
generally on
um trying to understand the the
anatomical and functional structure of
the
the brain and its expression in in
mental processes
um
so i hope that said kind of
um i'm sure it's deeply unfair both to
the people i've
i've quoted and many others who i
haven't
um but it's an attempt to say to give a
sense of
what informs my thinking
and and and do pick me up on things that
you think
um are mistaken or um
naive um later um
i want to talk about the real world now
which is we've already had mentioned
um i thought ashish was comment earlier
that
the thing about science is i think it's
the thing that keeps you honest
i thought that was spot on and that's
certainly
um where i come from in the world as it
really is
it seems to me has always been the thing
that keeps
did i lose connection
iris
sorry i lost connection for a moment or
maybe
i i didn't hear the last uh 20 seconds
oh i could be here
so should i should i pick up from here
sure thank you okay so
the starting point for me is is the real
world which
a lot of people talk about increasingly
as a kind of challenge to
to ai generally
and some people even
kind of view it as a set of kind of
ecosystems that
humans and non-human intelligences
inhabit um
uh i've heard a lot in this discussion
these discussions
about the view of the universe as
essentially a very very large
collection of probability distributions
and that their
preferred method for achieving both
benevolent
and risking malign ais
is to search though those distributions
to find dependencies which will guide
behavior in that universe
i don't know whether this is
idiosyncratic but i would say cognitive
science research suggests that
statistic elements are present
in in in the cognitive sciences and
certainly an important part of
experimental another
methodology but they're essentially
second order and
most uncertainty is uncertainty in the
agent's
uh head uh not uncertainty in the world
i don't know whether i should quote the
einstein cliche
god does not play dice but despite the
great levels of uncertainty
uh i think most of that uncertainty is
it is ours
not the universe and that's even true in
medicine
which is of course um
a field which depends hugely on on
statistical and other methodologies
and probabilistic bayesian and other
methods now i'll say a bit more about
that
but i'll suggest that medicine is
actually just another
complex ecosystem um with its associated
theories
um just a little bit tiny bit of history
this is a rather famous man called jj
gibson
um who many years ago wrote a book
on visual perception and and his um
it's still in print and it's still uh
you can still get it
um but it's a very sort of traditional
uh psychological work um
but what he was interested in which we
don't understand vision
whether it's he never thought about
robots but certainly in humans and other
animals
you need to understand um the
the the geometry that what he would say
the perception of the world at a point
um and how it changed as the
uh as as the entity moved around its
world
um more familiar perhaps to you as is
neil and simon
um the great influences of me
um who were they were first to kind of
use chess
as a uh as a tool
an ecosystem i'll claim um to
investigate human problem solving
but you know clearly there is a
well-defined
deterministic game here
and that's still
informs a lot of ai but um
because of the scale of the search
problem
involved has been seems to have been
particularly
picked up on in
in the post singularity
conversations um i like to
just to kind of make it slightly more
concrete um
um i played far too much freestyle i
don't know how many of you do
um i'm not a great player by any means
but somehow rather it
it comforts me as i'm trying to think
about hard things
um and um it's evident to me that
that you know i've learned whatever
skills i have
um using what i think would be obvious
to all of us
you know a whole variety of different
representations which are
spatial logical a bit a bit of
probabilistic reasoning of a um a rather
simple
kind um there's strategic things
um making opportunities making space
um knowing things that can i can push
things and there'll be a cascade
there's lots of different things and but
one thing's very striking of course is
that there's lots and lots of local
symmetries
so that the same theory or the same
method
can be applied in all sorts of
situations so that
the combinatorics of this are far
smaller than their peer
and and and i hope somebody will correct
me um but it seems to me that that's
true of go and chess
too which are games that people have
made a huge amount of capital out of in
terms of the
complexity of search spaces um
that ais have to operate in
and learn over and and and
you know you can play go with a nine
nine by nine board probably
even smaller than that and it's
essentially the same game and even
within the small game there's huge
numbers
so that actually the true it seems to me
and and i'm very willing to be corrected
it seems to me that
that that this is over it's inflated
the the claim that these are real
challenges um
not the people do them as well as um
alphago um uh there are lots of reasons
for that but but that the games
themselves are
somehow representative of the of the the
kinds of challenges that we
face in our uh real world ecology
um the most recent book i've mentioned
is by brian cantwell smith
reckoning in judgment the promise of ai
and he
essentially has the same intuition that
i have which is that
um human
decision making reasoning all the other
cognitive
skills we have um depend on many
different kinds of representations many
different little theories
um and um and this is actually
although clearly a very
unpopular may not be the right word but
not widely accepted in the
safety community excuse me
um this is actually a better solution
um than just blind search for
statistic learning statistical
dependencies
and applying those to to kind of compute
um uh expected value
um i think i would stick my neck out and
say
that not only is that basically true but
that effect will increase with the scale
and heterogeneity of the ecology
so as we try to throw ais the things
which
humans do
because complex like meds
will find that the effect of gets bigger
so let's say a little bit about medicine
uh
you know people have been trying to
apply ai techniques for
as long as i've been in the field to
medicine um
and with the the recent explosion
uh of interest in machine learning
um and so on we're seeing again another
series of books focused on medicine with
practitioners
writing books taking views on whether
they think ai is a good idea or not
um i'd like to suggest medicine is a
complex ecosystem
and probably among the largest most
complex domains of human knowledge and
practice
yeah i i can't think of a more complex
difficult one it's science base
is vast and continues to grow
quickly and again as with
the silly example of free cell um
it's informed the practice of medicine
is is informed by
lots of kind of theories about causality
and
statistical dependencies structure
geometry anatomy time biochemistry i
mean
that list will go on a long way and and
human doctors carry a large amount of
that information
um in their heads these are
representations which do
good work in different situations
uncertainty pervades this pervades
medicine
in every way you can think of um
but it's not kind of uncertainty really
in the world
the real world knows what it's doing
fairly deterministically
the uncertainties in our heads epistemic
eternity
so just very quickly um a bit of history
um ai in the clinic the first person who
applied
bayesian methods um there's a man called
tinder dumble the cardio cardiovascular
surgeon
in uk um who
um wrote paper in 1972
after he built a little bayesian
diagnosis system
um and made himself very unpopular with
his colleagues
um because he showed that this what he
used to dismissively call
idiot base um could outperform
even the most experienced um
and
senior doctors in the specialty
tim helped us we built another system
for also for diagnosis just a single
single task what's wrong with this
person um
and again another bayesian system um and
which the
doctor had the computer in front of her
and looking she could look at the screen
on the right which has
information she talked to the patient
collected information
put it into the computer and it updated
the probabilities
in the in the left here bottom left of
the
only five up five decision options
um and um uh
that was a an interesting result but it
was even then
in 1980 or thereabouts it was obvious
that this
statistical approach from purely
engineering or practical point of view
was a black box
and the clinicians were very
uncomfortable about it as they are today
as you know a huge debate
um picking up a
quote from stuart russell um he makes
the remark the machine viewers
might as well arrive from outer space
our chances of controlling
a super intelligent identity from outer
space are roughly zero
um uh i mean that might be true of a
superintelligent
entity i suspect he's absolutely right
um
though i think our doctors in fact had a
even though they didn't really had no
probabilistic training
that they did have a reasonably sensible
intuitive
um uh says what the numbers meant
in a very simplistic way um uh
and said he makes the argument that many
are making
that to creating ai systems that
guarantee we won't understand them
um are not a good idea
and i'm sure everybody here you know is
familiar with
this concern what it did though was
lead us to kind of move on and say well
can we build ais that are more
human-like
and i'm going to jump forward about 20
years
um or more um and this is a view
of a what's called a multi-disciplinary
team of
uh in the royal free hospital in london
um
who look after all the patients with
breast cancer or suspected breast cancer
um it's it's a major hospital
um you can see a lot of people there are
oncologists surgeons
radiologists pathologists nursing staff
and others all with specialist knowledge
different pieces of the
of the cancer and breast cancer
ecosystem and they kind of come together
uh um in order to kind of decide what
um interventions they think should be
offered
to their patients um i
probably can't really see this but what
this is is just a view of the kind of
for a particular patient um
a summary of the data that we have about
that patient which has been collected
before the meeting
um and then it applies
a i would claim as a human-like
cognitive agent model
to make a set of suggestions for each of
those suggestions what what should be
done whether it's
chemotherapy or or a genetic
assessment or a variety of things there
are many others
the system knows about 250 different
variables that potentially can be
relevant
and then on the basis of information
which is less than that much less than
that
about a particular patient it will then
give some recommendations to do
together with the arguments for it
against uh
and and link back into the into the body
of knowledge
uh that supports those the reasoning
um um so that so
i'll just briefly i'll give some more
evidence that this approach was quite
successful but
um despite that being a fairly major
center for breast cancer
um the application i didn't make it
clear that was
projected on the on the front of the of
the multi-disciplinary
meeting room
about 93 compliant with um
with national guidelines in the uk and
the
machine made about 97 so it was a
small but important improvement
um a separate piece of work with
something called peter hammond
um we looked at clinical safety and and
how
do people describe their strategies for
managing safety
uh within breast cancer and indeed many
other cancers and there's a
um basically a lot of little mini
theories or skills you know that what
people do is they prevent
things they look try to prevent things
that that are
known and likely to happen um they
optimize the actions the secrets of
actions they take
they monitor the patient um and so on
these very simple
rules um but actually
probably i'm not sure whether
stop mumbling i'm not sure unless you're
not sure whether that
the performance can be improved on by
but other methods because there isn't
actually that much uncertainty in me
in the practice um
so i'd like to give you a bit of bit of
intelligence a bit of a bit of evidence
that this kind of general approach
is is is worth considering
uh and and i think is on the road to
contributing to
some sort of super intelligence in the
future um
in in in not nick's book he says
that clearly the most advanced ai system
is far below the human baseline on any
reasonable metric of general
intellectual ability
um this is a set of just a summary table
of a set of
reports in in peer-reviewed medical
journals
that use the meth that we've developed
um and as far as i know there are no
published
trials uh that don't improve upon
uh the performance of human clinicians
so i think this these very strong
statements
um although they're very kind of widely
accepted
um i i think need to be kind of
discussed
um i think to me it's fairly clear
um that we could build
now uh a general purpose medical
ai that could do a very large amount
of what human medical professions do
and do it better
so um coming back to this question of
convergence and um progressiveness
um so in mike waldridge's book which i
strongly recommend he says something um
again something that i can't quite
accept which is that you know basically
we don't know anything about mind and
consciousness intelligence
you know how these things evolve how
they work um
they're utterly mysterious um
and the conclusion is is that this
fundamental lack of understanding makes
strong ais so difficult to approach we
have no idea where or how to begin
i don't think these statements are true
i think
if you spend some time you know
i'm not recommending anybody anybody
does it i've been it's been a
bit of an obsession of mine but trying
to kind of understand
what's happening in in in the cognitive
sciences and whether they're
um it's converging um they're converging
on on a
small number of theoretical frameworks
um i think you might find there's more
known
than you would conclude from this
statement and in fact one of the things
that
really struck me when i wrote when i
read um
[Music]
the super intelligence was that he gave
a set of
possible pathways to superintelligence
he's he he he didn't have the example of
somebody will come up with a theory
um and and i'm not gonna make a
prediction
but i think it's coming
so is the agi safety program progressive
by which i mean um
in as a cognitive scientist is and
focused on human like ai
is empirically grounded as we said
earlier
is it is is it is it keeping us honest
are we in the real world do we have a
kind of
strong formal mathematical normative
rational whatever your favorite word is
theory is it useful and controllable
and are the different theories which
there are still many
coming together what's not clear to me
um is whether a safety program
wants to be progressive in any of that
sense or even if it wants to be
progressive according to different
criteria um
and i'll just put a few in there and and
you may correct them
um again as i said earlier this debate
seems to be going on
through a series of of
i would say parties and books i think
this is the people who write these books
are very knowledgeable very capable
and they're trying to present their
positions
um but that i'm not aware of a
clear form of formal process of
peer review and all the rest that's
typical of science
um if anyway if we are going to try and
sign up to how we
how can we attempt to be programmatic or
stuart suggests for some possibilities
um i'm not quite sure even having read
the relevant chapters what that really
translates into
but you know it still doesn't leaves me
with
you know i'm not sure it's grounded
which is the word i've used
two or three times in these meetings um
or that the experiments that people are
doing and i'm
focusing on on the
compute as the as a potential solution
to
real world challenges i'm not sure
whether these things have been validated
in the sense that most scientists would
who are worried about pseudoscience
would accept
um and i do think it would be helpful
if the community is engaged with the
many other communities who are
worrying about the the theoretical and
and scientific problems
um so just to be provocative
i'll close with a
suggestion for how we might develop a
general program for making ais safe
from a cognitive science perspective
the first thing i noticed there is no
safety theory i haven't come across
anybody
that correct me of course but who
actually even says what safety is
it just relies on in the common sense
meaning of that word
but it appears to be strongly
linked to the concept of alignment which
has a
technical meaning
i'm not aware of any kind of formal work
on
making such critical decisions based on
a clear understanding of safety
um and there are certainly are working
in logic
develop two non-classical logics that
would be relevant in that context
described in the book i mentioned um and
and from a practical point of view i
don't think anyone would be surprised to
say we need to
have good methods of design um and ways
of proving
that they have certain properties like
safety properties but many other
properties too and there are
again in software engineering lots of
well not lots but some
um kind of quite quite versatile
techniques like model checking
um the next stage is to kind of learn
is deploying these
designs um and and i would recommend
that we learn contingencies in the
ecosystem
not just in one representation like
probability which i think will prove to
be
limited value but in many
representations
many which are obvious um
the ais should be designed to
continuously monitor and predict
obvious hazards and re-plan anything
they're doing against
constraints and the hammond rules for
safe
uh management of patients on
chemotherapy are
a kind of simple example of that so
there's a lot to do
and and finally um in terms of how do we
control
um these ais and i'm very mindful that
um the assumption is that
the ai's are in principle open-ended we
don't know what their capabilities are
um and so control is a huge problem
for that but there are already you know
proposals around
uh for ais to monitor their own and be
built to monitor their own behavior
and predict unpleasant consequences
and the final thought i'll leave you
with um
is which i'm working on myself is that
whenever a medical ai
an autonomous medical ai makes a
decision
we could be enforced that ai has to
consult
any number of other ais with
capabilities in
other parts of the ecosystem um
medical specialties and so on and may
not
act um without
authorization from the the community of
agents
including humans so
uh i not sure whether you will find that
satisfactory but i hope it's a
point of discussion thanks very much
john so there were a number of questions
posted in the chat
and i think if i wrote down correctly
now i can't find
it was ashes who will have the first
question
otherwise we'll go to i had the second
question
um when you said that
you foresee um black box approaches
as being not as applicable
the more complex the main groups and
foreign
there was a bit of background could you
um could you repeat the introduction
right i conducted myself can you hear me
now yes
right um i was interested when you said
that you believe that white box
approaches that are
pretty much designed by humans would be
superior
to black box approaches to ai where you
just have a model
and a bunch of numbers and the ai gives
you what you want
um how do you
how do you consolidate that world view
with the existence of something like
gpt3
which is probably the best tool we have
for for understanding language today in
terms of an ai
and it was basically made by you took a
model you gave it
all of the internet and it it gave you
the ability to
predict the next words in a good
sentence
yeah i think that um
the all sorts of kind of issues around
what i said i can i can see and we're
not going to kind of cover them all
today but
that you use the word um gbt3
understands the language i don't think
that's a common
commonly accepted um being able to
predict the next word
is a technical trick um
being able to have a wide ranging
conversation um
is what most people i thought in in
computational linguistics
would expect as a minimum
so so being able to predict or give
something is calibrated
for one specific skill let's call it
that
very well calibrated it is seems to me
is a long way from being an agi well
what do we care if anyone
understands medicine if it can tell you
what treatment to use
i'm sorry if it can tell you what
treatment do you usually think that's
understanding medicine
uh because it only tells us what to do
at the moment
in very limited ways i mean people
rightly criticize early
expert system database systems for would
call them narrow ai
um my my
suggestion that for the wide range of
routine things that
human doctors nurses and other people
have to do
um is is you know it's a considering
significant level of generality of
expertise
but it's hardly scratching the surface
of what doctors actually do
and we're and and and we have still a
long way to go this is i'm not
claiming that either that that this
approach
is has converged then we have a general
theory i'm not claiming that
i'm saying that i think it is
progressive on a number of dimensions
um and um
you know if you remember the the
definition like tosh's definition
it is that we are developing theories
incrementally to cover more and more
of the available data um and
um i i mean i i guess your i don't know
what your background is ashish but but
medicine
is is often thought by
technical people to be much simpler than
it actually is
and i don't i don't think there's a
single
medical ai around that understands
medicine in the way i would use that
word
okay that makes sense thank you okay
i had a question um it's in the chat
i'll just read it aloud the tasks that
doctors engage in
seem much broader than just medical
diagnosis prognosis or treatment
like convincing patients to do what is
best for them having a good bedside
manner dealing with byzantine
bureaucracies etc
can we really make an ai that can
replace the doctor for all those tasks
right now no
that's the same answer but i think
that's a variant of the
of ashish's question um the uh
and you're absolutely right um that was
the point i was making that that
the actual delivery of looking after
other human beings
is a very complex subtle um
uh process um i'm
i don't want to be and so to say say
that this
problem any of these problems are solved
i'm saying we are
progressing in ways and i think we're
further down the road
than is assumed um
you know so i i if you're interested i
mean after what um
ibm um start the marketing department
started making
uh a fuss about what's and how it was
going to um
um revolutionize cancer care
um i wrote a little article which made
exactly the same point you just made
which is that medicine is a more
um it's a bit more complex than the ibm
marketing department seem to think
and and the main successes in it for
example
in medical ai these days are in image
processing of one card or another
which doesn't involve much human it's an
area where
the data are too overwhelming for human
beings that's generally recognized
um
and i think it's um there's a lot
to be done to break out of the the kind
of
you know we'll solve we've solved the
image processing problem
um into wider areas of medicine
okay so uh the next question is
from me where i would ask um if there
are any uh
concrete work concrete research papers
being done
in ai safety uh that you feel in
particular
are not progressive not uh moving the
state of the art forward
i'd love to know the answer to that
question sir and the reason i found this
community was because i was trying to
answer it myself
and as far as i could see um
nobody was um even interested it's not
only in ai it's software engineering
generally i mean there's a small
um uh and computer science there's a
there's a relatively small
software safety community um very
sophisticated but it's very but it's
very
um uh it it's very technical and it
certainly doesn't
um i and so and they and they basically
ignored
safety uh uh ai's safety completely
um so when i discovered there was this
community
where actually when i went to miri and
discovered there were people
interested and then suddenly realized of
course the problem they're trying to
solve is different from the problem that
i've been trying to solve
i've been trying to understand you know
how do we um
kind of achieve some sort of engagement
between these different worlds but no i
don't i can't point to at
i think if we did the proper uh
literature
search we'd find we'd find a few but i'm
not aware of any
uh anything very substantial since
subrata and i wrote book in 2000.
okay jack you had a question yeah
um so i've well i've got a couple but i
guess
the first one that i'll ask is um just
kind of as a
broad summary of uh of the talk you gave
right i guess i'd like to present like
just a
brief summary of what you said and see
if that seems to line up with what
you're trying to say fascinating
yes please okay so um
what it seems uh what i what i
got from your talk is that it seems like
um
ai safety is uh
or the ideas that rest behind ai safety
is not
um a very formal or scientific um
field as a whole and it's not resting on
the kinds of
principles that will let actual
scientific progress
get made um and
that's kind it doesn't seem like you're
not concerned about um
maybe having malignant in the eyes but
it seems like you're concerned with the
way we're going about
um trying to prevent bad outcomes
i'm asking i'm just asking a simple
question i'm new to your field
and i've done my best um to
kind of understand some of the emergent
themes and
what people are committed to there's
lots and lots i still don't know
um and and so you know
as as zoen asked me you know any papers
um i would ask i mean that clearly
there's a lot there's the lineman
newsletter there's a bunch of things
you know i mostly i have to confess that
you read these rather broadly
focused books and try to observe the
debate that's going on
um but i do accept absolutely accept
that
the kind of safety um you know post
singularity if
if that's a commonly used phrase
ai safety um that you're concerned with
is different um from the
kinds of things that i've been concerned
with for many years
um and i'm trying to find if there's a
way that we can
you know cooperate um
and and whether the participants i mean
i thought i thought
zoran's introduction about
pseudoscience was was right the
participants in this community
um do they share um the
kind of criteria that i i would
suggest for for a progressive research
program
um or or different ones
so that could be my next question could
you share your slides one more time and
go back to the particular slide
where we're talking about some criteria
for whether
ai safety or whether a science is a
progressive science on it because it
might be worth
taking at least a little into into these
to see whether
um we can say something about this
um so uh i think there's a
few slides further in um
so that was my attempt to summarize this
personal view you know what i think is
progressive ai research and then i put
up a few things which
criteria that the ai safety community
um might might also consider
relevant in judging their progress okay
so um prit that's a progressive research
program
oh sorry progressive research program
yes i'm sorry
so um grounded i i expect this is
uh both in something like some of the um
exact observations that we have for
instance we kind of
assume that uh ais will
just follow their their objective
function of a cliff
and then we go back into literature and
see has this actually happened and
we find examples and it looks like yeah
systems do this kind of thing so both in
indirect observations and also in a lot
of the other um
more strategic papers that the future of
humanity institute are making
they uh there's a lot of work trying to
put to refer
to refer back to existing strat existing
work on principal agent problems
and and things like that so okay
been glad with some guidance on what to
read there um
what i i realize now when i used the
word grounded i i was overloading the
word here partly was it empirically
grounded which you just said
uh some some of the work is is empirical
um but really the stronger thing i had
in mind is this side is it grounded in
the real world
um as an ecosystem that
that um the agent whether human or
non-human
um uh exists in and
and to my mind there's very little
grounding of that kind
so i think that would be my next
question ecocomplete
so could you explain what is uh i guess
ecosystem oh well you're entitled you're
entitled to criticize me i've just tried
to find a snappy
uh a snappy word um the
um uh what i mean is um
if we if if you want to build
um an ai system that is
capable of either supporting humans in
a particular medical domain such as
cancer
or operating autonomously
um is have you built an agent that
actually
has the sufficient knowledge
to cope with all the issues
objectives outcomes in that
ecology now a specialty of medicine is
not a very big
in human terms it's not a very big
domain
and clearly if you if you expand up
to the whole of medicine and then expand
up again to the hold of
human expertise um
we pro having a property called
eco-complete
is going to be just as hard uh to prove
as anything else i say i put it in there
to to
kind of stimulate uh some debate
rather than because i am proposing it to
the to the
ai safety community
fair enough so um
when i think about uh ecosystems in
in ai safety i think about something
like uh
for instance and and agi might be
created by the military and
in that case there will be some
stakeholders who are generals or might
be created by google
and then it might be doing something
else is that the kind of thing you're
you're talking about on it's a little
bit of a joke sir on things like np
completes them
you know that sort of thing but i'm
suggesting it's not just about
um computational power
in order to for an algorithm to
search or or um
to terminate on some arbitrary
objective it's that it has sufficient
knowledge and i might say understanding
of the ecology
and and and compute power is is not the
only
thing to consider there it's it's
content
okay uh we will just quickly go over the
the
the next two align compatible and
controllable
um so um is aich a progressive research
program
in that i think it's pretty clear that
we're striving for
figuring out how to make ai aligned and
controllable
that seems uh human companies that's
clear that's why i included them they
were
they were very prominent and so it
obviously had to appear there
okay so if it has a question or
okay sure so yeah this whole thing is a
question really
um i would say um that
i i'm still struggling a little bit
aligned
uh but with the word aligned and i may
not have understood it but it does seem
to be
in some sense i mean as in in in stewart
russell's case
um the the the the ai's
preferences are aligned with human
preferences
um and and that's i
i find that very surprising i mean human
and if human cognition isn't like that
we don't have a whole set of preface of
preferences
that then some ai can observers
and and learn and then
but i may not have fully understood his
point
okay there is a new question and let me
just i haven't followed along in the
chat so let me just see who
whose turn it is and that would be the
uh
that would be uh ali
uh principles for the field do you mean
principles like
bellman optimality or the principle of
least action or comes razor
are principles in control system physics
and science respectively
well disciplines have different
uh criteria um and
the ones you've described are not
particularly familiar to me they're
obviously come from some of the physical
physical sciences
and mathematical models i suppose
um i i again this is a question and i'm
not um
uh trying to tell you what your criteria
should be
um i'm just trying to understand what
they what they are and what the
community would take to be
a basis for judging progress
okay chris has a question what does
convergence mean
in a progressive research program and a
ui
so i guess we are back here at the
same slide converging questions
so what i mean by convergence in the in
the top bit and maybe
the cognitive sciences generally is the
you know there are probably hundreds of
thousands of active
that might be over over a large number
but active
uh cognitive scientists around the world
um hundreds at least and maybe thousands
of different theoretical
sort of perspectives um
and we always have to ask ourselves the
question
um you know are all those things in any
way coming together
so that we you know as they appear to
have done
in physics and chemistry and and many
other
kind of hardcore sciences um
and um and so
if if if
there if there are many different
approaches to
ai safety and maybe they're only a
couple i don't know
um then one again
asks well are they in lack of touches
sense are the each is each new theory
um uh better than the last or
or or combining the strengths of more
than
one perspective as has happened in
in classical sciences
yeah i think that that
that makes sense i'm slightly confused
because of often
um convergence often
crops up in the context of convergence
of
ai
values motivations and human values and
motivations but that's not
quite that's not what's what in other
words in the context of alignment
which yes yeah not what you meant to
that particular point yeah
yeah it's like all these unifications
that we've achieved in the various
sciences
okay i think the next question is for
you and then my video is
acting really strange when you use this
race hand
functionality i don't really know why
but it starts flickering
um so um i think it's better if people
write in the chat
that they have a question rather than
raise the hand anyway liam go ahead
uh it's not really a question it's more
of a comment but in terms of
um trying to find
convergence within scientific fields
um i feel like one of the problems there
is in terms of
data that uh you know
that the reason why in the 16th and
1700s you could have a
enormous boom in the physical sciences
or the natural sciences
is because the data is pretty especially
in physics the data is pretty limited
um they're very very simple systems that
you can um look at and experiment with
um and they're more or less
self-contained
um whereas in terms of something like
cognitive science like we're looking at
the the spectrum of humanity
it seems awfully difficult to have the
same kind of
um limits around the data like even the
question of like what
what counts and what doesn't count
whereas in within within
at least within newtonian mechanics
that's like a very simple
question of like where is the limits of
what you're looking at
um so i feel like in terms of things we
it could be a it could be a mistake to
want
um convergence within
you know the social sciences uh prior to
having
um uh the the sort of the large amounts
of data that
justify um a broad scale hypothesis or
theory of
of the human so to speak
well this is my intuition of the state
of affairs
i suppose data data for me is is is the
same as what
sir and i think use the word
observations you know you
and and as you say you know when
when newton and many other
sort of early physicists were doing
their observations on
on light and prisms and all the rest
i've forgotten all my
high school physics um uh
they they had the advantage not only
there wasn't much data
but also it was highly reliable
uh there wasn't much uncertainty in the
results and maybe it was harder than i
imagined
um but this distance but
with the big data revolution um
suddenly it appeared that you know with
probabilistic and statistical methods
with very large numbers of observations
we could detect
and learn dependencies
over large populations of people or
patients or
you know climate parameters
and everybody got very excited
but even within the big data community
um there were critics who were saying
look you know
they're just data they're just numbers
you know we
the fact that we can detect dependencies
doesn't mean to say we understand
anything
um and and i think then
that kind of i don't know whether i'm
right about this but did that
internal debate kind of got
forgotten when when the enthusiasm for
machine learning
appeared and people started saying oh
look you know we can
we can now we've got so much compute
power we can we can solve all these
problems
um just by data we don't have to
understand anything
and and and that's what you know some of
these books i've
uh mentioned on the way that people are
taking different positions on whether
that's true
okay i think uh can i do a follow-up is
that okay
sure um within
uh cognitive science in terms of uh
research that's going on
um is there much research uh
done in terms of um communities
or cultures which are
very very different from the west so i
don't mean very different in terms of
vietnam i mean like
extraordinarily different in terms of um
greenlanders or indigenous australians
or something like that
i honestly don't know liam um i'm
um and and that's my shortcoming i
i think science
until the recent um the anti-science
movement got going uh was western
science was
where it is because it seemed to be the
one
that delivers um but i think
that's also rather self-serving view of
western scientists
um and one way it's obvious
is that we can't make a strong statement
over strong statements about that or at
least i can't
um it is because we you know
we don't read chinese and we don't need
vietnamese and we don't read lots of
other things so
there may be lots of things going on
that we just don't know about
a friend of mine who's a chemist organic
chemist that spent half his year in
in in beijing and one of china um
and and it's absolutely clear that you
know
the chinese are using western science
from top to bottom um and the comp the
competition
is is is political
um i didn't mean in terms of methodology
i meant in terms of
in trying to construct a framework of
what the human mind is
oh we'll see in terms of uh
studying groups that because because you
know there's a
oh it seems well you did say cultural
yeah um yeah
well i mean i go back to saying i don't
know the answer
thanks turn off and then check and then
click on awesome
um so could you speak up a bit uh
maybe yeah is that better oh yeah that's
better
okay um so
[Music]
you mentioned earlier uh you mentioned
earlier that you were trying to think
about ways to
um maybe see if these two
different views of ai can sort of
cooperate
um and i think that's
i think that's a pretty good thing and a
pretty great reason to have a talk like
this
i'm also pretty new to this uh entire
field i've
only been studying this whole thing for
i don't know maybe four months or so
but one thing i do know is when you
talked earlier in your presentation
about
how black boxes are not really suitable
for
relying on or for
giving lots of responsibility to um i
know that
the um that's one thing that
i hesitate to say both of our
communities
agree upon and i know that at least
the ai alignment community has
has sort of empirical research going on
in this area and it's called
interpretability research where the
explicit goal is to take things that are
black boxes
and make it apparent either what the
process of reasoning is
or what the examples being used to
generate outcomes are
but to make the
the the process much more transparent in
what's going on and i feel like
i feel like when you look at some of the
recent developments you can see that
there is a
pretty clearly empirical trend in
in figuring out exactly how to gauge
whether or not
a prediction is trustworthy um or or
figuring out
certain kinds of reasoning from a model
which might only have
uh statistical correlations to go by
um that just sort of seemed like a
common point that i thought it would be
worth emphasizing
oh sorry i've missed i've obviously
missed something
key um i'm i'm very interested in this
conversation
but i'm also very conscious that i don't
know very
much about you know what the ai safety
community
values um and and
there will be lots of different views
i've looked at a number of
um youtube videos by young polsky and
kinski and and as well as all the other
ai heroes um or emery heroes
um but i'm i'm still not sure
just how many different perspectives
there are in berkeley is obviously
very influential um i think i think
there are some people in
in oxford who um are very
well regarded as well um a different
perspective sorry i'm i'm
i'm i'm going on um i
i only know my side of it and the
cognitive science side well
um i don't uh i i would need to work
with
with other people if we wanted to kind
of take this further
absolutely and that makes sense um and i
wasn't necessarily saying that i was
expecting you to know all those sorts of
things um okay i mean i i barely know
them myself
um i was sort of just trying to say that
uh some of these things which it seemed
like
uh uh you you
hadn't seen i'm just saying they they do
actually exist and they are out there
that's all i was trying to say okay
thank you that's great
and i think actually uh ali's question
is also
an example of this where ali is i don't
know if you can see the chat
window allen has a rather long oh it's
technically it's not a question because
there is no question but
a description of some kind of
improvement in the
the field of ai safety where we now have
a good
impact measure a mathematical strict
formal definition of
something that we previously only had
like a vague uh
description of and certainly no good
formal description of and
insights as an example of something that
is
um that is uh progressive in the in the
obvious sense
that we are making progress therefore we
couldn't uh formalize this and now we
have a
formalization that we are happy with
right
and and i'm sure there will be um a
number
of such advances
um and very some may be very narrow and
technical
and some may be philosophically deep and
wide
um and you know if
if we can kind of surface
things and might even
have some sort of a consensus across the
community
um that would be i think that would be
some historical importance actually
okay um i also have a question but i
should also say that we are at the 1.5
hour mark now
so uh that means that of course john if
you need to
leave or if you're tired or anything
like that then uh
please feel free to uh then we'll just
say thank you
thank you otherwise i think there might
be more questions
um and i'm i'm happy to stand happy to
say and thank you all for
the interest you've taken and and uh
look forward to
oh and if anybody wants to copy the
slides then
you can you can presumably post them in
some way they're
sure if you can just share them somehow
so if
i can have another question uh and
that's
um that was what you said right that you
had time for more questions
yeah yeah okay so um let me try to
find the place where our opinions
probably are
perhaps closer to each other so a
problem
with medical ai is de-skilling
in that if you get some ai that is
reasonably good at some medical task
then eventually you're going to have a
situation where people you know as
the people with the experience before
they
they lose the ability to uh perform some
of the tasks
that's you agree this is like a
absolutely
absolutely a problem that can happen
it's one of the most one of the
motivations for um
this kind of human-like ai approach
that i've has always been a
bit behind my research it is that
um if it's transparent in and
intelligible in human terms
and potentially can even tutor
users in human terms then then there may
be a
strategy in there for uh reducing
these skilling risks
and i i think we we agree on this and i
think
the the the the main a or part of the
ai safety issue is that
uh this is something that it appears
with things like gt3 like this is
something that
could easily happen that we could get
ais that are productive
and able to do things that previously
humans could also do but that are also
not able to tutor us that are not uh
gt3 is basically a black box and if we
see this now if it's just like
a small part of radiology that is
automated done by
ais then the d skill might not be a big
problem
but as we peer further ahead into the
crystal ball
then it appears that more and more will
have a greater and greater d skilling
and that seems extremely problematic on
the medium long term
the the uh possibly greater
uh these killing so that would be my
guess at where you and me are closest
do you agree that this is a conceivable
problem on the medium long term
well i think it's i think it's more than
conceivable it will inevitably happen
and we've lots of um kind of
issues that are going to emerge that we
haven't
recognized yet but these de-skilling has
happened with
well it's certainly been discussed with
countless other technologies
um and most famously with you know
calculators
um and um but i think
as far as i'm aware in most of those
cases
uh you know technical problems have
technical fixes
um um i bet you know so i think even
though
kids at school have
um calculators and allowed to use them
in all sorts of ways
um uh the curriculum has been evolved
and changed in order to compensate for
some of those
that those de-skilling things i'm not a
teacher
um so i don't really know whether i'm
right about that
um but um yeah i i would expect
uh as as i've kind of said in the uh
the the miri interview
um i i think quite a lot of problems are
soluble with well
well understood engineering or practical
solutions
um there's just gonna be a lot more that
we
haven't anticipated
okay um so uh in terms of
in terms of the d skilling uh if i can
just pick up on that quickly
that's all right um we have a situation
in australia that's
maybe vaguely akin to
ai de-skilling people
which is that our government prefers to
import skilled immigrants rather than
use our education system to
educate australians um and
and that's like led to all sorts of
really bizarre problems of like
erosion of communities in terms of
people can't get jobs
and also like a for students a
disillusionment with the education
system itself
um so i think i think soren's
question is is right on point of that
it wouldn't just be that um we would
lose
knowledge as individuals or whatever but
it has
really widespread effects um on the
society at large
sure that's right but in the case of
say excuse me radiographers
that seems okay to me rather like
um the example of driverless cars it's
where
the the benefits from improving
diagnostic skills are so great that
introducing ai into you know um
x-ray diagnosis interpretation of scans
and so on
um the case that would be irresistible
because
you know we want to get better um
diagnoses constantly and uh
if that you know meant that one class
of skilled people were now you know
worse off because
um they that their skills were no longer
valuable
i think society would have would would
willingly accept that
on a utilitarian calculus and likewise
with
driverless cars um if they are
really achievable achievable and uh well
being achieved already
um the the saving in lives
and the economic benefits are so great
and that um
it again it's probably um irresistible
and so that's bad luck for truck drivers
um
it's bad luck with people who enjoy
driving on on the highway and we're not
likely to
to put much weight on that um when the
price is
you know much human life so there are
some areas where
i i think we'll
we are not going to
we're we're not likely to to think that
that's a loss we're going to think that
it's a net gain
and so the question is where along the
line
do we start to notice a loss
of value as opposed to a gain
well as you know
mcking has commented a couple of times
before um
one route to net loss
is the the kind of thing that we can do
for human benefit building
benevolent ais i can see all kinds of
ways they can be weaponized
at scale
what sort of weaponizing do you have in
mind just in the straightforward
military sense or or
yeah yeah i mean um
i mean the kind of theoretical framework
that kind of underpins all our work
is i believe
um actually has a very broad range of
capabilities and
um although it's not embodied in the
sense of a robot
we don't build robots we build
disembodied applications for medical
purposes
um uh i think there is a framework there
which could be used to build a cognition
engine
for for physical robots and
vehicles and humanoid robots and all the
rest i mean i
i have no no convincing evidence of that
i just
uh believe it it is the case and
therefore you could pick up a lot of
this technology
and add value to um existing
weapon systems um
military vehicles and um and uh
and and so on and um they could do a lot
more than they can do at the moment
but you but please treat that with um
with
as evidence-free you know as a scientist
i'm not making you claim i'm just saying
i worry about it
um i guess i feel less worried if i
think
whenever i can think of a clear ethical
principle
that can be laid down um
and that could be accepted
internationally then
you know i'd i would feel safer so for
example
um the principle that
if you use an ai uh say a
in a call center to to to respond to
people
um if if you make it um
part of the code of practice that this
thing would source announce itself
as being a digital system to the to the
person it's talking to and never
um never never pass itself off as a
human being
that seems to me a clear um
[Music]
code a clear principle that you could do
in the code of practice
similarly i this is too this is perhaps
too utopian but um ideally
um in the case of autonomous weapons
you had a principle where
an ai was assisting a human being in
controlling those weapons
as opposed to human beings theoretically
supervising
sort of really autonomous systems
that would be a kind of safeguard i'm
thinking of the analogy with
with in the case of airliners
which are very largely flown by um
by computers nowadays um
there are sort of two two philosophies
that you can work on one in which you
have
um a plane that's flown by computers and
is supervised by human beings
which sounds like a great idea except
that you
realize that human beings easily get
tired
are lazy and dodge the rules and so on
uh and and lose their skills
their employers are likely to to just
not train them properly
because they're just minding the
machines uh versus the opposite
philosophy
having human beings flying actually
flying
the airliners hands-on with um
computers in the background supervised
supervising them but not in the sense of
giving them orders but watching them
sounding the alarm and so on so that
human beings are forced
to be active and vigilant
and skillful
that seems to be a clear sort of
principle that you can lay down if you
know if you can do that for
um airline pilots maybe you can do it
for um
for soldiers in the um generals in the
field
make make sure that they are the ones
who are
choosing the targets fighting the
battles and being assisted by ai as
opposed to the opposite
but of course in the military field that
but it's very idealistic because
you know it's like long may idealism yes
i fear i live in a slightly darker world
than you
with the the dozen or so
gangsters that uh a recent american
president
wanted to um hook up with um
don't think like that at all
yes and i mean it's like it's like
having the the it's like laying down the
principle for safe at land mines but
um uh land mines have been able to go
into certain principles such as
self-destructing after one year um
that you must always map exactly where
you put lay your land mines and so on
and so forth
a great principle guiding principle but
obviously
in in the heat of in the heat of real
war
um they will end up being used
indiscriminately
so if i can just uh check onto chris's
airline
analogy where um one of the um
the ways that people are trying to avoid
this de-skilling
is by having explicitly that
the the aircraft is indeed capable of
major aircrafts are capable of
basically landing themselves in
inclement weather and
that means that what what they do in
practice is that
the uh the pilot is actually doing the
the um the landing supervised by by the
computer
so the computer is training the uh the
human
in in how to uh in in how to try to land
this ally
and then um for very difficult uh
landings then the uh the airline can say
the aircraft can say okay i'll do this
because there's literally zero
visibility
so the plane will just land itself um
but uh
you know but you can actually prevent
this de-skilling
by a a conscious effort and that's been
done
what is currently being done by with
airline pilots
wasn't it sometimes the opposite that in
some difficult situations
ai is not very capable in landing and
humans
need to do it and that's the problem
that when humans
lose their skills in the easier
situation
then they also never evolved to the
skill that is required from them in the
difficult situation
but the ones who still have the skill
uh trained for difficult situations they
do better
than ai in this situation
i think this uh actually a something
that has been a
difference between uh europe
and some of the the countries in uh
in asia because in in europe there are
very stringent regulations about
how much airline pilots need to train
and in some countries there are
substantially less
um this of these requirements which
means that
the uh for instance something with
training on
landing with the civil landings a
european pilot could expect that he
would
be forced to do a lot of these learning
so you can get a lot of experience
where whereas if you're flying in
indonesia
the the rules are a lot less strict
which means that
unfortunately if it's easier for the
pilots then they can just ask the plane
to
to uh to land but on the other hand they
get less experience and less training
and of course also less time in
simulators and these kind of things
um which also mean that the uh
the safety record is better in europe
than in indonesia
this is not my speciality by the way i
should say
the planning systems i'm working with
for airlines are the non-safety critical
systems so um i'm not 100
sure
john i believe in that um in that
interview
that you gave as a reference for this
discussion
you mentioned that you
you've been involved in in um
for uh formal proofs of of
software safety i think you you referred
to
either that well
i've collaborated with many people
including
computer scientists who call themselves
formal methods
people and um so i'm not i'm not the
person who does the proofs by any means
um it's uh the um
that the book contains quite a lot of
formal material but that was mostly
written by
um by bo sabrata um
you know and as i say i'm i'm an
experimental scientist originally by
training who
you then who then moved into ai
in about 1973 um
and i've still well i'm going on
no i didn't i mean there is work that
could be done i'd be very disappointed
how little work that that has been done
since
since we've kind of finished that that
line of work
the line of research
well there are people who sort of um
attempt
formal formal proofs
oh there's massive amounts in the area
of safety and think of
in ai specifically and i don't really
know how
how much that is feasible or um
well well i plea please point me at
that the things that you found because i
i
i haven't and i'm really interested
he has lots of pessimistic arguments
about the possibility of achieving
ai safety and i don't know whether they
make out just formal
formal proofs or formal work i don't
know whether he's one of these sources
i'll see um but but it's not anyone i
know much about you see that's why i was
yeah yeah no i'm out of date
so uh you mentioned uh uh
in the uh in the first part of your
slides
and it seems uh one of his
points is that he cares a lot about the
demarcation
problem between science and student
science
and i know you're not really saying that
aict is a pseudoscience but
how close are you to saying that ai
safety is a pseudoscience
um
i will not take a position because i
don't think i know enough
about what all the people are doing um
at this point i
i i get the impression that most people
in in the field don't don't regard
the things that trouble me as very
troubling
so i may simply be misguided
um uh i just wanted to ask john what do
you find
most troubling in terms of like ai
safety what's your like
do you have a number one concern
um
i'm mentally trying to kind of pick one
out um and and i think i end up
saying that um
you know i'm not a philosopher of
science my mickey mouse theory
of scientific method is that it's a
mixture of
empiricism and formalism and
pragmatism um and
most of what i have read um
by significant figures in
in the field um are
very much narrower than that in their
concerns
you know i think
but my i i'm going to try and wriggle
off this hook
because the last thing i want to do is
give the impression that i'm
i'm criticizing at this point i'm not
criticizing i'm trying to understand
you know what what people think they're
doing and i thought that
zurin's introductory question about
pseudoscience was was was really great
um because it me it means i assume that
nobody wants to be accused of doing
pseudoscience
so i'm keen to
see what the field strategy is to avoid
it
because because conventional science i
think has got that down
they know how to do that it's not
perfect but it
and i don't think that i'm not aware of
discussions of that kind
within the ai safety
world
in that interview you um drew an analogy
with
um
with medicine now with with drugs
medical drugs specifically specifically
and i suppose other i suppose you were
also thinking about other medical
procedures um and you're saying that
you know we we accept that there's a
duty of care
by um pharmaceutical companies
and by all all doctors and all medical
practitioners of any kind
um there's a duty of care and a
necessity for them to be regulated
externally and you're saying that
this should apply just in the same way
in
yeah not only in software in general but
also in in
ai products in general yeah i think i
think that's probably would be the
answer to liam
that putting my head on my hat
on as a practitioner in ai
um that's the thing that worries me is
that
um people aren't having these
discussions in conventional ai um
[Music]
and and at some point it's going to get
resolved um but most of the public
debate
around ai is is
is is not as kind of focused and and
practical as that
it's about you know
general principles of whether you can
make a machine as
smart as people or you know whether you
can create a super intelligence by
any of the means that have been proposed
um
so i think liam that's my um partial
answer
um that um i i i worry
and nothing's changed as far as i can
see since 2014
uh i worry that um uh
this isn't really on on on the research
agenda
for ai generally agi
i'm sure there's lots of essays and
papers and so on and so forth but i'm
not
uh most most people seem to be
writing or reading books which are kind
of big stories
rather than actually would i say doing
the work
was it last week we were saying that um
we need
a few disasters um of
less than existential level
um to wake people up to the dangers of
yeah
so careful what you wish for my thoughts
exactly
so now we're at the two hour mark so now
i'll repeat of course that
uh thank you for your answers john and i
don't know if you know and have any
final questions but you're we we've
certainly appreciated
uh the discussion thank you yes i will
slip away now
for me wonderful oh yeah yeah yes thanks
very much
and i i i hadn't really realized until i
read
uh read that you know the materials but
we had quite such a heavy weight um in
uh
in in our group i mean i thought i'm
sure with lots of heavy weights in
different ways but i
didn't know about your personal
expertise
can i retired a retired heavyweight
might be better
okay guys liam do you have a final
comment
oh yeah yeah i do i do if if that's okay
in terms of like uh you know the the
melding between
science and uh and ai where like
uh we're we're basically assuming that
ai is going to like have massive
influence on individuals and groups
lives and stuff and so we want to have
some idea in terms of alignment we want
to have some idea
of how to negotiate that in a positive
way
right at least not negative um
and part of my concern with
studies like cognitive science is asking
questions effectively what is the human
that if we're not if we're not uh
taking into account groups that are not
western these cultural groups are not
western and how they think how they
what their preferences so to speak if
you know if that's a real thing
ah then we're we're effectively going to
be creating
ai that that prejudices one way of
living
um significantly over others i think
that's one of my greatest fears
going forward in the field i understand
do you think there's much hope that
cognitive science will mean
i mean well you know we're primates
we do what primates do and that doesn't
matter really
which particular culture whatever we
come from um
you you know primates have a repertoire
of
behaviors and activities and
we develop various kinds of institutions
to manage them
and um i think most
most of the guidance i'm not going to
say that i'm not qualified to say it
you know i just think um there's lots of
social scientists and anthropologists
and psychologists
cross-cultural psychologists and all
sorts of people
you know who who look at big questions
whether there's any people
who are cognitive scientists or ai
people who look at these kind of
cultural questions
i simply don't know
okay all right thanks okay yeah thanks a
lot bye
see you next week where we will discuss
chapter three of
roman yampovsky's uh aisf safety
schedule system table
okay thanks everybody see you next week
thank you very much