The promise of AI with Demis Hassabis - DeepMind: The Podcast (S2, Ep9)

welcome back to the final episode in
this season of the deep mind podcast and
boy have we covered a lot of ground
from protein folding ais to sarcastic
language models sauntering robots
synthetic voices and much more it has
been quite the journey
but we do have one more treat in store
for you a chance to hear from deepmind
ceo and co-founder demis hasabis the
outcome i've always dreamed of is
agi has helped us solve a lot of the big
challenges facing society today be that
health creating a new energy source so
that's what i see as happening is a sort
of amazing flourishing to the next level
of humanity's potential with this very
powerful technology
this was my opportunity to ask demis all
the things that have popped into my head
during the making of the series
well most things we'll see how far i can
push it
as luck would have it the day i sat down
with demis coincided with the opening of
deepmind's sparkling new premises in
london's king's cross
there weren't many people about yet so
it felt like an exclusive preview i feel
like i'm in a
high-end furniture catalogue
let me set the scene for you this new
building is rather beautifully appointed
it's got a double helix staircase
running through the middle
there are fiddle leaf trees in
practically every corner
and there are stylish fluted glass
critter doors between offices
and yes those meeting rooms christened
after great scientists galileo ada
lovelace leonardo they are all still a
[Music]
at feature
sparkling push the boat out while
sipping on my beverage of choice some
memorabilia outside demise's office
caught my eye a nod to alphago's famous
victory over lisa dole in the game go
there is sitting underneath two
extremely fancy black spotlights
a chessboard in a black frame and if i
go over to it
there's a picture of gary kasparov
the
legendary chess player who was beaten by
deep blue the ibm computer
he signed the chessboard and it says for
the alphago team keep conquering new
heights i mean
just a chessboard designed by kasparov
on the wall perfectly standard
oh awesome oh we're going in
hi
great to see you
after settling down inside demise's
office i started by asking him about
deepmind's long-term vision of building
agi or artificial general intelligence
it's an ambition that has been baked
into deep mind's dna from the very
beginning
i think it's fair to say that there's
some people in the field who don't think
that agi is possible
they sort of say that it's a distraction
from the actual work of building
practical ai systems
what makes you so sure that this is
something that's possible i think it
comes down to the definition of agi so
if we define it as a system that's able
to do a wide variety of cognitive tasks
to a human level that must be possible i
think because the existence proof is the
human brain
and unless you think there's something
non-computable in the brain which so far
there's no evidence for then
it should be possible to mimic those
functions
on effectively a turing machine a
computer and then the second part of
that which is it's a distraction from
building practical systems well i mean
that may be true in the sense of what
you're mostly interested is in the
practical systems agi itself is a big
research goal and a long term one it's
not going to happen anytime soon but our
view is that if you try and shoot for
the stars so to speak then any
technologies that you sort of build on
the way can be broken off in components
and then applied to amazing things and
so we think striving for the long-term
ambitious research goal is the best way
to create technologies that you can
apply right now how will you
recognize agi when you see it will you
know it when you see it
what i imagine is going to happen is
some of these ai systems will start
being able to use language and i mean
they already are but better maybe you'll
start collaborating with them say
scientifically and i think more and more
as you put them to use at different
tasks slowly that portfolio will grow
and then eventually we could end up it's
controlling a fusion power station and
eventually i think one system or one set
of ideas and algorithms will be able to
scale across those tasks and everything
in between and then once that starts
being built out there will be of course
philosophical arguments about is that
covering all the space of what humans
can do and i think in some respects it
will definitely be beyond what humans
are able to do which will be exciting as
long as that's done in the right way and
you know there'll be cognitive
scientists that will look into does it
have all the cognitive capabilities we
think humans have creativity what about
emotion imagination memory and then
there'll be the subjective feeling
that these things are getting smarter
but i think that's partly why this is
the most exciting journey in my opinion
that humans have ever embarked on which
is i'm sure that trying to build agi
with a sort of neuroscience inspiration
is going to tell us a lot about
ourselves and the human mind the way
you're describing it there is though
this big goal in the future that you
steadily approach
i'm wondering whether
in your mind there's also like a day
where this happens like you know how
children dream of lifting the world cup
have you thought about the day when you
walk off
walk away from the office and you're
like it happened today
yeah i'd have dreamed about that for a
very long time i think it would be more
romantic in some sense if that happened
where you you know one day you're coming
in and then this lump of code is just
executing then the next day you come in
and it sort of feels sentient to you be
quite amazing from what we've seen so
far it will probably be more incremental
and then a threshold will be crossed but
i suspect it will start feeling
interesting and strange in this middle
zone as we start approaching that we're
not there yet i don't think none of the
systems that we interact with or built
have that feeling of sentience or
awareness any of those things they're
just kind of programs that execute
albeit they learn but i could imagine
that one day that could happen you know
there's a few things i look out for like
perhaps coming up with a truly original
idea creating something new a new theory
in science that ends up holding maybe
coming up with its own problem that it
wants to solve these kinds of things
would be sort of activities that i'd be
looking for on the way to maybe that big
day if you're a betting man then when do
you think that will be so i think that
the progress so far has been pretty
phenomenal i think that it's coming
relatively soon in the next you know i
wouldn't be super surprised the next
decade or two shane said that he writes
down predictions and his confidence on
them and then checks back to see how
well he did in the past do you do the
same thing i don't do that no i um i'm
not as methodical as shane so and he
hasn't shown me his recent predictions i
don't know where they were secretly
putting them down i have to ask him it's
just a draw in his hand yes exactly
[Music]
like shane legg deepmind's co-founder
and chief scientist who we heard from in
an earlier episode demis believes that
there are certain abilities that humans
have but are missing from current ai
systems
today's learning systems are really good
at learning in messy situations so
dealing with vision or intuition in go
so pattern recognition they're amazing
for that
but we haven't yet got them
satisfactorily back up to be able to use
symbolic knowledge so doing mathematics
or language even we have some of course
language models but they don't have a
deep understanding yet still of concepts
that underlie language and so they can't
generalize or write a novel or make
something new how do you test whether
say a language model has a conceptual
understanding of what it's coming out
with that's a hard question and
something that we're all wrestling with
still so we have our own large language
model just like most teams in these days
and it's fascinating probing it you know
at three in the morning that's one of my
favorite things to do is just have a
have a little chat with the uh
with the ai system
uh sometimes but i'm generally trying to
break it to see exactly this like does
it really understand what you're talking
about
one of the things that suspected they
don't understand properly is
quite basic real world situations that
rely on
maybe experiencing physics or acting in
the world because obviously these are
passive language models right they just
learn from reading the internet so you
can say sort of things like alice threw
the ball to bob ball through back to
alice alice throws it over the wall bob
goes and gets it who's got the ball and
you know obviously in that case it's bob
but it can get quite confused sometimes
it'll say alice or so it'll say
something random so it's those types of
you know almost like a kid would
understand that and it's interesting are
there basic things like that that it
can't get about the real world because
it's all it sort of only knows it from
words but it's a that in itself is a
fascinating philosophical question i
think what we're doing
is philosophy actually in the greatest
tradition of that trying to understand
philosophy of mind philosophy of science
when it's 3am and you're talking to a
language model do you ever ask if it's
an agi yeah
i think i must have done that yes with
varying answers but it has responded yes
at some point yeah it does sometimes
respond yes and you know i'm an
artificial system and it knows what agi
is
to some level i don't think it really
knows anything to be honest that would
be my conclusion it knows some words no
words a clever parent yes exactly
for the moment at least ai systems like
language models show no signs of
understanding the world
but could they ever go beyond this in
future
do you think that consciousness could
emerge as a sort of natural consequence
of a particular architecture or do you
think that it's something that has to be
intentionally created
i'm not sure
i suspect that intelligence and
consciousness are what's called double
dissociable you can have one without the
other both ways my argument for that
would be that if you have a pet dog for
example i think they're quite clearly
have some consciousness you know they
seem to dream they're sort of self-aware
of what they want to do
but they're not you know dogs are smart
but they're not that smart right and so
it's my dog isn't anyway but on the
other hand if you look at intelligent
systems the current ones we built okay
they're quite narrow but they are very
good at say games i could easily imagine
carrying on with building those types of
alpha zero systems and they get more
general more and more powerful but they
just feel like programs so that's one
path and then the other path is that it
turns out consciousness is integral with
intelligence so in least in biological
systems they seem to both increase
together so it suggests that maybe
there's a correlation it could be that
it's causative so
it turns out if you have these general
intelligence systems they automatically
have to have a model of their own
conscious experience personally i don't
see why that's necessary so i think by
building ai and deconstructing it we
might actually be able to triangulate
and pin down what the essence of
consciousness is and then we would have
the decision of do we want to build that
in or not my personal opinion is at
least in the first stages we shouldn't
if we have the choice because i think
that brings in a lot of other complex
ethical issues tell me about some of
those well i mean i think if an ai
system was conscious and you believed it
was then you'd have to consider what
rights it might have and then the other
issue as well is that conscious systems
or beings have generally come with free
will and wanting to set their own goals
and i think um you know there's some
safety questions about that as well and
so i think it would fit into a pattern
that we're much more used to with our
machines around us to view ai as a kind
of tool or
if it's language based the kind of
oracle it's like the world's best
encyclopedia right you ask a question
and it has like you know all research to
hand but not necessarily
an opinion or a goal to do with that
information right its goal would be to
give that information in the most
convenient way possible to the human
interactor wikipedia doesn't have a
theory of mind and maybe it's best to
keep maybe it's best to keep it like
that exactly okay how about a moral
compass then can you impart a moral
compass into ai and should you
i mean i'm not sure i would call it a
moral compass but definitely it's going
to need a value system because whatever
goal you give it you're effectively
incentivizing that ai system to do
something
and so
as that becomes more more general you
can sort of think about that as almost a
value system what do you want it to do
in its set of actions what you do want
to sort of disallow
how should it think about side effects
versus its main goal what's its top
level goal if it's to keep humans happy
which set of humans what does happiness
mean we can definitely need for help
from philosophers and sociologists and
others about defining and psychologists
probably you know defining what a lot of
these terms mean and of course a lot of
them are very tricky
for humans to figure out our collective
goals
what do you see as the best possible
outcome of having agi the outcome i've
always dreamed of or imagined is
agi has helped us solve a lot of the big
challenges facing society today be that
health
cures for diseases like alzheimer's i
would also imagine agi helping with
climate
creating a new energy source that is
renewable and then what would happen
after those kinds of first stage things
is you kind of have this sometimes
people describe it as radical abundance
if we're talking about radical abundance
of i don't know water and food and
energy how does ai help to create that
so it helps to create that by unlocking
key technological breakthroughs let's
take energy for example
we are looking for as a species
renewable cheap ideally free
non-polluting energy
and to me there's at least a couple of
ways of doing that one would be to make
fusion work much better than nuclear
fission it's much safer that's obviously
the way the sun works we're already
working on one of the challenges for
that which is containing the plasma in a
fusion reactor and we already have the
state-of-the-art way of doing that sort
of unbelievably the other way is to make
solar power work much better if we had
solar panels just tiling something you
know half the size of texas that would
be enough to power the whole world's
uses of energy so it's just not
efficient enough right now but if you
had superconductors you know room
temperature superconductor which is
obviously the the holy grail in that
area if that was possible suddenly that
would make that much more viable and i
could imagine ai helping with material
science that's a big combinatorial
problem huge search space all the
different compounds you can combine
together which one's the best
and of course edison sort of did that by
hand when he found tungsten for light
bulbs but imagine doing that at
enormous scale or much harder problems
than a light bulb that's kind of the
sorts of things i'm thinking an ai could
be used for i think you probably know
what i'm going to ask you next because
if that is the fully optimistic utopian
view of the future
it can't all be positive when you're
lying awake at night what are the things
that you worry about well to be honest
with you i do think that is a very
plausible end state the optimistic one i
painted you and of course that's what i
reason i work on ai's because i hoped it
would be like that on the other hand one
of the biggest worries i have is what
humans are going to do with
ai technologies on the way to agi like
most technologies
they could be used for good or bad and i
think that's down to us as a society and
governments to decide which direction
they're going to go in do you think
society is ready for agi
i don't think
yet i think that's part of what this
podcast series is about as well is to
give the general public a more of an
understanding of what agi is what ai is
and what's coming down the road and then
we can start grappling with as a society
and not just the technologists what we
want to be doing with these systems you
said you've got this sort of 20-year
prediction
and then simultaneously where society is
in terms of understanding and grappling
with these ideas
do you think that deep mind has a
responsibility to
hit pause at any point
potentially i always imagine that
as we got closer to the sort of gray
zone that you were talking about earlier
the best thing to do might be to pause
the pushing of the performance of these
systems so that you can analyze down to
my new detail exactly and maybe even
prove things mathematically about the
system so that you know the limits and
otherwise of the systems that you're
building at that point i think all the
world's greatest minds should probably
be thinking about this problem so that
was what i would be advocating to you
know the terence towers of this world
the best mathematicians is actually if
i've even talked to him about this i
know you're working on the riemann
hypothesis or something which is the
best thing in mathematics but actually
this is more pressing i have this sort
of idea of like almost uh avengers
assembled of the scientific world
because that's a bit of like my dream
deterrence tower agree to be one of your
avengers
i don't i didn't quite tell him the full
plan of that
i know that some quite prominent
scientists have spoken in quite serious
terms about this path towards getting
agi i'm thinking about stephen hawking
do you ever have debates with those kind
of people about what the future looks
like yeah i actually talked to stephen
hawking a couple of times i went to see
him in cambridge i was supposed to be a
half an hour meeting but we ended up
talking for hours
he wanted to understand what was going
on at the coalface of ai development and
i explained to him what we were doing
the kinds of things we've discussed
today what we're worried about and he
felt much more reassured that people
were thinking about this in the correct
way
and
at the end he said i wish you the best
of luck but not too much
then he looked at right in my eye and
twinkle in his eye like it was just
amazing that was literally his last
sentence today
best of luck but not too much
that's lovely that was perfect it is
perfect
along the road to adi there have already
been some significant breakthroughs with
particular ai systems or narrow ai as
it's sometimes known
not least the deepmind system known as
alpha fold which we heard about in
episode 1.
alpha fold has been shown to accurately
predict the 3d structures of proteins
with implications for everything from
the discovery of new drugs to pandemic
preparedness
i asked ms how a company known for
getting computers to play games to a
superhuman level was able to achieve
success in some of the biggest
scientific challenges in the space of
just a few short years
the idea was always from the beginning
of deep mind to
prove our general learning ideas
reinforce learning deep learning
combining that on games tackle the most
complex games there are out there so go
and starcraft in terms of computer games
and board games and then the hope was we
could then start tackling real world
problems especially in science which is
my other huge passion and at least my
personal reason for working on ai was to
use ai as the ultimate tool really to
accelerate scientific discovery in
almost any field because if it's a
general tool then it should be
applicable to many many fields of
science and i think alpha fold which is
our program for protein folding is our
first massive example of that and i
think it's woken up the scientific world
to the possibility of what ai could do
what impact do you hope that our fold
will have
i hope alpha fold is the beginning of a
new era in biology where computational
and ai methods are used to help
model all aspects of biological systems
and therefore accelerate our discovery
process in biology so i'm hoping that
it'll have a huge effect on drug
discovery but also fundamental biology
understanding what these proteins do in
your body and i think that if you look
at machine learning it's the perfect
description language for biology in the
same way that maths was the perfect
description language for physics and
many people obviously in the last 50
years have tried to apply mathematics to
biology with some success but i think
it's too complex for mathematicians to
describe in a few equations but i think
it's the perfect regime for machine
learning to spot patterns machine
learning is really good at taking
weak signals messy signals and making
sense of them which is i think the
regime that we're in with biology
how could ai be used for a future
pandemic
so one of the things actually we're
looking for now is the top 20 pathogens
that biologists are identifying could
cause the next pandemic to fold all the
proteins which mean you know it's
feasible involved in all those viruses
so that drug discovery and farmer can
have a head start at figuring out what
drugs or antidotes or antivirals would
they make to combat those if those
viruses ended up mutating slightly and
becoming the next pandemic i think in
the next few years we'll also have
automated drug discovery processes as
well so we won't just be giving the
structure of the protein we might even
be able to propose what sort of compound
might be needed so i think there's a lot
of things ai can potentially do and then
on the other side of things maybe on the
analysis side to track trends and
predict how spreading might happen
given how significant the advances are
for science that are being created by
these ai systems do you think that there
will ever be a day
where an ai wins a nobel prize
i would say that just like any tool it's
the human ingenuity that's gone into it
you know it's sort of like saying who
should we credit spotting jupiter's
moons is it his telescope no i think
it's galileo and of course he also built
the telescope right famously as well as
it was his eye that saw it and then he
wrote it up so i think it's a nice sort
of science fiction story to say well the
ai should win it but at least until we
get to full agi if it's sentient it's
picked the problem itself it's come up
with a hypothesis and then it solved it
that's a little bit different but for
now where it's just a fairly automated
tool effectively i think the credit
should go probably to the humans i don't
know quite like the idea of giving
nobels to inanimate objects like larger
hadron collider can have one exactly
regression
telescope can have one exactly i just
quite like that idea
even before agi has been created it's
clear that ai systems like averfold are
already having a significant impact on
real world problems
but for all their positives there are
also some tricky ethical questions
surrounding the deployment of ai which
we've been exploring throughout this
series things like the impact of ai on
the environment
and the problem of biased ai systems
being used to help make decisions on
things like access to healthcare or
eligibility for parole
what's your view on ai being used in
those situations i just think we have to
be very careful that the hype doesn't
get ahead of itself
there are a lot of people think ai can
just do anything already and actually if
they understood ai properly they'd know
that the technology is not ready and one
big category of those things is very
nuanced human judgment about human
behavior so parole board hearing would
be a good example of that there's no way
ai's ready yet to kind of model the
balance of factors that experience say
parole board member is balancing up
across society how do you quantify those
things mathematically or in data and
then if you add in a further thing which
is how critical that decision is either
way
then all those things combined
mean to me that it's not something that
ai should be used for certainly not to
make the decision at the level ais at
the moment i think it's fine to use it
as an analysis tool to triage like a
medical image but the doctor needs to
make the decision
in our episode on language models we
talk about some of the more concerning
potential uses of them
is there anything that deep mind can do
to really prevent some of those
nefarious purposes of language models
like spreading misinformation
we're doing a bunch of research
ourselves on you know the issues with
language models i think there's a long
way to go like in terms of building
analysis tools to interpret what these
systems are doing and why they're doing
it i think this is a question of
understanding why are they putting this
output out and then how can you
fix those issues like biases fairness
and what's the right way to do that of
course you want truth at the heart of it
but then there are subjective things
where people from different say
political persuasions have a different
view about something what are you going
to say is the truth at that point so
then it sort of impinges on like well
what does society think about that and
then which society are you talking about
and
these are really complex questions and
because of that this is an area i think
that we should be proceeding with
caution in terms of deploying these
systems in products and things
how do you mitigate the impact that ai
is having on the environment is there
just a danger of building larger and
larger and larger energy-hungry systems
and having a negative impact yeah i mean
we have to consider this i think that ai
systems are using a tiny sliver of the
world's energy usage even the big models
compared to
watching videos online all of these
things are using way more computers and
bandwidth second thing is that actually
most of the big data centers now
especially things like google are pretty
much 100 carbon neutral but we should
continue that trend to become fully
green data centers and then of course
you have to look at the benefits of what
you're trying to build so let's say a
healthcare system or something like that
relative to energy usage most ai models
are hugely net positive and then the
final thing is we've proven is that
actually building the ai models can then
be used you know to optimize the energy
systems itself so for example one of the
best applications we've had of our ai
systems is to control the cooling in
data centers and save like 30 of the
energy they use you know that saving is
way more than we've ever used for all of
our ai models put together probably so
it's an important thing to bear in mind
to make sure it doesn't get out of hand
but i think right now i think that
particular worries are sort of slightly
over hyped
while demis and his colleagues at
deepmind are thinking hard about what
could go wrong when ai is deployed in
the real world what really shone through
during our conversation was demus's
faith in the idea that ultimately
building ai and agi will be a net
positive for the whole of society
if you look at the
challenges that confront humanity today
climate sustainability inequality
the natural world all of these things
are in my view getting worse and worse
and there's going to be new ones coming
soon down the line like access to water
and so on which i think are going to be
really major issues in the next 50 years
and if there wasn't something like ai
coming down the road i would be
extremely worried for our ability to
actually solve these problems but i'm
optimistic we are going to solve those
things because i think ai is coming and
i think it will be the best tool that
we've ever created
in some ways it's hard not to be drawn
in by demise's optimism to be enthused
by the tantalizing picture he paints of
the future
and it's becoming clearer that there are
serious benefits to be had as this
technology matures but as research
swells behind that single north star of
agi
it's also evident that this progress
comes with its own serious risks too
there are technical challenges that need
resolving but ethical and social
challenges too that can't be ignored
and much of that can't be resolved by ai
companies alone
they require a broader societal
conversation
one which i hope at least in some small
way is fueled by this podcast
but i'm struck most of all by how far
the field has come in such a short space
of time at the end of the last season we
were talking enthusiastically about ai
playing atari games and go and chess
and now
all of a sudden as these ideas have
found their feet we can reasonably look
forward to ai making a difference in
drug discovery and nuclear fusion and
understanding the genome
and i do wonder what new discoveries
might await when we meet again
[Music]
deepmind the podcast has been a
whistle-down production the series
producer is dan hardoon with production
support from jill ateneku the editor is
david prest sound design is by emma
barnaby and nigel appleton is the sound
engineer
the original music for this series was
specially composed by elainey shaw and
what wonderful music it was
i'm professor hannah fry thank you for
listening
you


AI Safety Reading Group (Session 39)

hello and welcome to the 39th session of
the AICC reading group and today we are
going to talk about an article called
politics with up streams of AI by
Raymond Brenner I couldn't find a
picture of Raymond brand on even though
I'm sure it's quite a bit and but it's a
writer and write a website called the
future prenatal with the sub time wiser
counsel for the smile sort of bro that's
like me as kind of a half and make the
horizon photos oh well it's a group
block em that call themselves new
reactionary and something called the
dark enlightenment enlightenment and I
couldn't find it really either pick that
Uncle Scrooge at the picture for a
reactionary that's not probably a really
good example and I'm not really sure
what this dark enlightenment really is
so I think we should probably focus on
the article itself oh yeah I was
thinking if and something to do with the
all right so actually in the discussion
afterwards which won't be recorded and I
have a some thoughts about whether this
can in any reasonable way they said to
be all right our new reactionary or have
anything to do with that but that's
probably a contentious socket so I one
included in the video and the main
metaphor of the article up streams is
like a river flowing from somewhere to
somewhere and this means that politics
in some which dominates in terms how
science is made how science what science
what's gold signs have and the the
paradigms and in in a very real sense
also the scientific methods that I used
and that will start sucking some
examples where that there's been a great
degree of state control over science
the first example is listen collision
which was a scientific theory about how
to grow agriculture that it was the gain
deal of political favor in the Soviet
Union and because it was very very wrong
it helped back the agricultural
production in the Soviet Union and a lot
of people stopped the two other examples
are connected Germany which were
realized that a lot of the physics will
make by were discovered by Jews and
because they couldn't stand that Jews
would be able to invade something that
the Germans couldn't that one that that
Aryan people couldn't then they try to
reinvent physics without relativity and
all these things and that went very very
poorly and they also had a cosmological
theory that was even that was decidedly
more crazy some guy some German had a
dream vision where he dreamed rent that
the moon was made entirely of ice and
used that to it for very very many
things about about the world based on
the limited tickets I got a skype sound
I don't know if someone else came online
and negated and we didn't hold on ok
little quick easy not lady ok I'll go
back so the last example is from of the
state having a great degree of control
assign is from a project that the United
States did in trying to predict and
control Latin America called Project
Camelot and the later política which was
more predicting and less controlling and
the reason why this is interesting is
this is a military project where that an
extremely Pele was it called a hierarchy
where the
let the general and chop from the army
representing the state and then the
scientists are explicitly below that
taking orders from the from the military
command both of these projects were very
small and ultimately quite insignificant
so the power relations that most AI
safety really think about is with some
researchers who are trying to control an
AI and the point of this article is that
the researchers are not just controlling
the AI as a state and on top of that is
also controlling with researchers and
through those Dai and how can the state
control well in two ways they can be
deliberate control obviously if it's a
state funded project that will be very
very real control if there's a promising
private project it would be a national
life and the state has a very clear
mandate from the people to to control
interesting technology control
technology with security implications
and the state will almost certainly the
remand Brennan doesn't really give an
occupied process in his analysis it is
clear that the state will participate
has decided to participate in an arms
race and so in this way the it's
impossible to oh it's very difficult to
foresee and AI projects that will not be
state a I at least in some in some sense
the other big problem is that could be
misalignment and the researchers say we
are building this project that will do
this in this and the blue cards here
something else it's also possible that
states are not unified and so one
department has this idea about what to
do with strong AI and another has some
other things about this and states are
really made of a can be seen as made up
of particular sections with Vice control
and finally a the state the people
web the colonists they might simply must
understand what what AI really is and
what is capable of so from this is
concluded that AI researchers need to
study how this power relation works and
in order to account for the LA I state
consequences so then the question that
is often raises whether math can save a
I and methods of course not mathematic
like this but something like if you're
building an AI that extrapolates human
values last time we talked about Iliad
Kowski here on the right career
extrapolated position which I found this
wonderful image this is the sum of
happiness so I think this is a really a
real nice picture and and and the
question is whether this will be
implemented and Richard Brennan is
negative about this because the state
will have some kind of political
priorities that will supersede and
override even if Alicia Elliott asking
on Mira was a close to being able to
implement and AI would create extra
bleep addition they would be
nationalized and the mathematical even
if it's a very beautiful and simple and
you could call a career an extra belated
relation even if that was completely
perfect then the state might still be
very very able to to come in and
override it and and from this Richard
Brennan concludes that a serious AI
project has has a big task in ensuring
its ideological integrity that it's not
hijacked either by insurance agents or
external like the agents like the state
and this doesn't even have to be very
very explicit and over like a
nationalization is very very over but
the programs are building vai where they
also get allowed their ideas and way too
soon from the world is through through
the media and the books and and think
that the state controls in some way and
this means that if the AI the people
building VAR and some way intellectual
net knees then it would be much there
will be very much under control of the
state and this is a problem and that
where which are brands will conclude
that the intellectual background and
moral education of the people who are
building this AI is very critical
section that's us gathered here gives
you two pictures garbage in garbage out
that if the AI researchers just if you
met him at the character the only we're
looking at Fox News or something like
that then they might build an AI that
really likes Tom Trump oh and I don't
know that compass in my example
yeah doesn't yeah I said Here I actually
made a rather great mistake that I'm
going to assault on the discussion
because here I took in a contemporary
political example when it was not
necessary and that's generally
considered a bad thing I shouldn't do
that I'm uh sorry I shouldn't do that so
I'm catching like the the key to this
moral education its historical data on
human values according to Richard
Brandon I think here we get closer to
some of the dark enlightenment values
that was very large emphasis on the
historical and that's the way you
understand the world ever they an
example of where AI could be really
problematic would be a sub today I if
you're met in the Soviet Union creates
and artificial intelligence then how
could we would be expected to be
friendly would we expect it because it
was created in a mathematical intersect
way that it would then bring about
paradise in the world or or heaven close
to I mean it's quite possible that
Joseph Stalin he she said some nice
things about the workers internationally
but what his true actions were very very
nice human aligned to be frank and and
this is a this required nothing you AI
researchers are able to in some way with
just the styling is dialing all of them
to build a nai and this seems like a
very very tall order for to figure out
that it's necessary to resist your
systolic and to actually do that so I
the claim is in particular that
political conflicts are some of the
things that makes state-controlled
really really dangerous and problematic
things like you have an AI China have an
AI and these two are against each other
it doesn't American AI and a Chinese a I
couldn't making them in a very
antagonistic relationship and this this
could cause an arms race and this caused
some values and a behavior that is very
very non human friendly so this kind of
thing has happened at some well I
difficulty with a brand a number of
reasonably comparable things have
happened in a previous time dimensions
yeah the Napoleonic Wars the world wars
communism Manhattan Project the cuban
missile crisis at example where this
that AI researchers really really need
to learn from and and of course this is
a different situation because technology
doesn't develop or yourself yes as
Richard random right and this means that
once we get to have strong AI and this
it will not be a directly comparable
situation to versus the world wars but
it is to human agents and institutions
that have a crucial role in this and it
is a mistake to think that is just an
algorithmic problem that is a mistake to
think if we get the details of Korean
extra belated position correct then
we're home free the problem is that this
is not might not might very well not be
vai that it's dope it will be a
politically neither okay people so we
have five moderately concrete
recommendations that I this is kind of a
summary study history of state influence
and technology and particular these
examples that he
mentioned before as well three Manhattan
projects the Manhattan Project is of
course also and a hugely Rachel case and
learn a lot from history in particular
about human values and morality and
figure out if you have an AI project
figure out what are the political
threats and defend against these both
internally and externally and study
political history behind a technological
development that's the article thank you
for listening I'll now stop the
recording and we'll go to the discussion
for the reason for the business on
YouTube see you next week


DeepMind: The Podcast (S1 trailer)

Fry: Coming soon, a podcast series
that gives you the inside track
on how artificial intelligence
is being created
and how it could change our lives
and the society that we live in.
person: All the sudden,
you've got a vast number--
literally a number of options
that is in the billions.
person: Results are amazing.
I think they're jaw-dropping.
person: The rules and heuristics
don't get better over time, but AI does.
person: I think it's gonna be
the most amazing
transformative technology
that humanity's ever invented.
Fry: For the past 12 months,
we've been tracking
the latest work of scientists,
researchers, and engineers at DeepMind.
robotic voice: Go up.
Fry: Ready to go?
person: I'll try to teach you
to make a sound.
Fry: And you're training me,
essentially, with reinforcement learning
and my reward function
is getting you to be happy.
Meep, and beep, boop.
person: Boop, bleep. Ahh!
Fry: And boop.
person: I'd go with the first one.
Fry: Welcome to "DeepMind: The Podcast."
I'm Hannah Fry.
I'm a mathematician who has worked
with algorithms for almost a decade.
Just in this series,
we've looked at energy conservation,
medical diagnosis,
and protein folding,
all of which certainly show
the machine's ability
to solve hard problems.
person: We've been able
to make a step change
on a hard problem that's been worked on
for over 50 years.
Fry: Of all of the results
that've come out of DeepMind,
this is the one that's got
the scientific community most excited.
person: Yes.
person: The sort of opportunity
to explore and search the space
is really something that's well designed
for AI to do.
It's a very efficient search algorithm.
person: Amazingly, we discovered
that the system
which had learned completely for itself
without a single piece of human knowledge
ended up being far stronger.
Fry: [gasps] I didn't know that.
We're looking at how
they're approaching the science of AI,
and some of the tricky decisions
that the whole field is wrestling with.
I think it's important
that you are intentional
about why you're building this.
And if you start from that premise,
then I think you're more likely
to do the good
that you hoped you were going to do.
person: So we don't just want
human imitation.
We want superhuman capabilities,
but without unsafe behavior.
Fry: But if anyone has an idea
of what it will take,
it's Demis Hassabis,
the CEO and co-founder of DeepMind.
Hassabis: All the big questions,
you know, the meaning of life,
how did the universe start,
what is consciousness--
all these questions, which I feel like
a blaring claxon in my mind
that I would like to understand.
And my attempt at doing that
is to build AI first.
Fry: So whether you want to know more
about where technology is headed,
or want to be inspired
on your own AI journey,
then you've come to the right place.
Hassabis: I hope that what people
are going to get out of this series
is a better understanding
of artificial intelligence,
and I hope they also
get a great feeling
for how exhilarating an endeavor
and a journey that we're on here.
Fry: "DeepMind: The Podcast"
with me, Hannah Fry.
Coming soon to your podcast provider.


AI and neuroscience: The virtuous circle - DeepMind: The Podcast (S1, Ep1)

I think I was about seven years old when
I wrote my first line of code it was
probably something simple printing my
name to the screen or a two-dimensional
shape that could be twisted and
stretched by my sequence of carefully
typed letters and digits it was an
extraordinary feeling a first sense of
where the logic defining power of the
computer could take us but it wasn't
until much later that I started to come
across the term artificial intelligence
or AI and well what a world that would
open up AI holds enormous promise for
the future and I think these are
incredibly exciting times to sort of be
alive and working in these fields we
want to kind of understand and master
increasingly complex systems AI must be
build responsibly and safely and used
for the benefit of everyone in society
and we have to ensure the benefits
accrue to everyone you know I think AI
can be one of the most exciting and
transformative technologies we'll ever
invent that is the voice of demis
hassabis the CEO of deepmind the
london-based
artificial intelligence company for
demos ai will allow us to create
computer systems that can learn to solve
complex problems by themselves in his
words society could use intelligence to
solve everything else cancer climate
change language energy in short to
advance scientific discovery but just
how far-fetched are these goals can
researchers really crack intelligence
and just how much of an impact would
that really have I'm Hanna Frey and this
is deep mind the podcast for the past
year I've been at deep mind HQ in London
for an inside look at the fascinating
world of AI research and where it's
going we will be telling you the
fast-moving story of the biggest
challenges in artificial intelligence or
AI so whether you just want to know more
about where the technology is headed all
want to be inspired on your own AI
journey then you've come to the right
place we will focus on the projects that
scientists researchers and engineers are
actually working on how they're
approaching the science of AI and some
of the tricky decisions the whole field
is wrestling with at the moment and
whilst we're here we've explored the
rooms full of computer screens where
scientists run their endless experiments
the meeting rooms where people write
intricate equations on whiteboards
packed to the rafters the robots and the
laboratories where banks of repetitive
robot arms grapple with piles of plastic
bricks and we've talked to a huge number
of people to try to understand what is
driving this new frontier the voices
that you'll hear in this podcast are
from the people that are at the cutting
edge of AI and machine learning and
quite a few of them are talking about
their work publicly for the very first
time but if we want to solve
intelligence let's start with a
fundamental question of AI what exactly
do we mean by intelligence if we're
trying to make machines intelligent what
are we actually aiming for this is sort
of something that's debated a lot in AI
world is like well do we want to you
know have our AI agents act exactly the
same way that people do like should they
be exactly human-like intelligent or
should they just be intelligent in
general
this is Jess Hamrick a research
scientist at deep mind her specialism is
imagination and mental simulation
there's sort of I guess like you know
one group of people who like to say that
you know we want to build something
that's just generally intelligent that's
really able to solve a lot of different
problems in the world that humans aren't
necessarily able to solve that has an
intelligence that's higher than humans
so this might be able to solve problems
like how do we cure all diseases like
maybe maybe and artificial intelligence
might be able to help us solve this
problem and that's you know something
human society and human civilization
hasn't yet been able to accomplish but
then there's also another group of
people who say that it's really
important for us to build AI that is
similar to human intelligence at least
in some ways I would consider myself to
be sort of in the latter group why does
it need to be similar to human the
reason is because as we build AI we as
humans need to be able to interact with
AI and collaborate with it be able to
understand the predictions that it's
making or the recommendations that it's
making and if we build AI in a way that
is maybe we are able to build AI and
it's generally intelligent but it acts
in a way that's so alien to humans that
we just can't really understand what
it's doing and I think that actually
would be a really bad scenario to be in
because either it means that people
don't trust it and then people are very
unwilling to you know use the
recommendations of this AI maybe it says
oh do this one thing and this will like
cure this disease and but people don't
understand why it's making that
recommendation maybe we miss out on a
lot of opportunities to really do a lot
of good in the world
we need Rai to understand the world in
the same way that we do it needs to be
able to explain itself to us so we can
be sure that we can trust it take for
instance the story of an AI that was
trained to diagnose skin cancer by
looking at photographs of skin lesions
taken by dermatologists the algorithm
did a good job of correctly labeling the
images but the researchers soon
discovered that the AI wasn't looking at
the cancer at all to make its decision
it had simply learned that lesions
photographed next to a ruler are more
likely to be malignant
not exactly trustworthy it's crucially
important that artificial intelligence
is able to grasp the subtleties of human
thoughts we want it to do what we mean
it to do not just what we say we mean
but that doesn't necessarily imply it
nice to think in exactly the same way as
people do there can be drawbacks to
trying to imitate human or animal brains
too closely we get into discussions
about where the strategy can limit you
this is Matt Banach Matt is the director
of neuroscience research at deep mind
where he draws on his experience in
cognitive neuroscience and experimental
psychology Matt believes the human mind
is the inspiration but AI research has
to take things further in its own way
you know the Wright brothers when they
solved the problem of flight you know
that people like to say oh they solved
the problem when they stopped trying to
copy bird's wings which you know in some
technical way might be true but they
wouldn't have gotten to where they were
right if they hadn't spend an awful lot
of time and if other people hadn't spent
an awful lot of time looking at bird's
wings and noticing the airfoil pattern
and thinking about the the dynamics of
the of the air that flows around an
object with this shape so yeah we we do
believe that we can look to the human
brain in the human mind for inspiration
but we also talk about them when the
moment comes where we need to kind of
step away from that and just build
something that does what we want it to
do
so what is the neuroscience equivalent
what are the birds wings of our brains
the aspects of our own intelligence that
we can use for inspiration as we build
AI well one area that seems to hold a
lot of promise is memory and in
particular something we all do known as
we play replay as a phenomenon that was
discovered in a part of the mammalian
brain the medial temporal lobe including
the hippocampus where you see neural
activity that suggests that past
experiences are being replayed
especially in in navigation for example
a rat will go through some environment
and a particular pattern of activity
will arise as it goes through the
environment and then later if you have
electrodes in the hippocampus you can
see that the same pattern of activity
the same sequence is occurring
suggesting that a memory is being
replayed of that experience and that
idea now has a firm place in AI if you
lose your car keys you can run your mind
through where you've been to workout
where you might have left them well I
first went into the kitchen and took my
coat off in the hallway put my bag down
on the side and oh yeah they're in my
back pocket that ability to replay your
experiences and learn from that memory
after the fact is a key part of what
researchers want AI to be able to do
here's more from that the way that
that's implemented in deep minds agents
is it's not exactly what you find in the
brain it wasn't as if people were trying
to slavishly recreate the the biological
mechanisms but the idea of replay which
was inspired by neuroscience came in
handy
in 2015 replay
played a pivotal role in a famous
deepmind breakthrough the team managed
to build an AI system that could play
arcade classics to a superhuman level
the old Atari games like space invaders
pong and breakout the AI use something
called deep reinforcement learning but
behind the scenes it kept a memory of
moves it made as it played and how those
moves had impacted on the final score by
replaying those memories the AI could
learn from his experiences it could work
out what sequences of moves worked well
which were mistakes and find strategies
that otherwise wouldn't have been
obvious but there's more to our human
memories than just a giant database of
facts of course you can remember the
name of the capital of France but you
might also be able to remember jumping
on the bouncy castle at your 6th
birthday party or the pranks you played
on your last day at school this is a
phenomenon called episodic memory and
it's something that holds a great deal
of promise for AI we talk a lot about
something called episodic memory which
is simply the cognitive ability to
retrieve a memory of something that
happened to you before we started
recording we were joking about like what
did you have for breakfast
your ability to cast your mind back to
that moment when you were eating
breakfast and retrieve that information
that's a function that psychologists and
neuroscientists refer to as episodic
memory and we have this category both
because psychologists work hard over
decades to fractionate memory into
particular domains or kinds but this is
a pretty high-level idea it's not like
replay it's just hey there's such a
thing as episodic memory which is very
important for human intelligence
maybe our agents should have episodic
memory what would that mean what would
it mean for an artificial agent to have
episodic memory
this is an intriguing possibility an AI
that can transport itself back in time
and recall entire events and experiences
rather than just facts when you stop and
think about it this ability to link one
memory with another is an amazing human
skill and if researchers can get a
better understanding of how our brains
actually do this it could be replicated
in AI systems giving them a much greater
capacity for solving novel problems
let's think about how that works for a
moment imagine that every morning you
see the same man in his thirties walking
a boisterous collie then one day a
white-haired lady who looks like the man
comes down the street with the same dog
with those events stored as episodes in
your mind you might immediately make a
series of deductions the man and the
woman might come from the same household
the lady maybe the man's mother or
another close relative perhaps she's
taken over his role because he's ill or
busy we weave an intricate story of
these strangers pulling material from
our memories together prioritizing some
pieces of information over others to
make it coherent it's something that's
been the focus of recent research by the
neuroscientists here a study in
September 2018 demonstrated the critical
role of the hippocampus that shrimp
shaped seat of memory in the middle of
the brain in weaving together individual
memories to produce new insight Jess
Hamrick is also looking at another way
that a eyes can be made to respond more
flexibly to new situations she takes our
inspiration from a different human
ability mental simulation what you and I
might call imagination imagine that
you're on a beach you'll have like this
mental picture kind of spring to mind of
you know mine at least is maybe a sandy
beach the bright blue ocean maybe some
palm trees on the slope
and so this is an example of what we
would call mental simulation it's like
we're mentally stimulating this picture
of the beach and then you can do things
with that simulation so you can imagine
adding other people to your imagination
you can imagine what would happen if you
like threw a ball if you're playing
volleyball or something like that so
these these sort of mental simulations
are really interactive and really rich
and I think that they underlie a lot of
our human ability to understand the
world and make predictions about the
world I should pause for a moment here
to explain what Jess and maps mean by an
age in here it's a word that's used a
lot at deep mind remember when people
are talking about artificial
intelligence they're really just talking
about computer code with the freedom to
make its own decisions and an agent is
just the noun that they use to describe
the part of that code that has agency
Jeff's is hoping to build agents that
are flexible enough to adapt to all
manner of environments it's a very grand
ambition but one with real potential to
see why let's go back to the RKO and
like game of space invaders mastered
using deep reinforcement learning to
create an agent called deep Q network or
dqn dqn was really sort of an amazing
technological feat because it was able
to be trained to play many many
different Atari games directly from
perception from pixels this is something
that hadn't been done before
but the way that dqn also works is that
it really just goes directly from inputs
to output so it takes in the image of
the video game and outputs immediately
what actions should be taken to maximize
the score in that game so maybe it's a
move left maybe it's you know push the
trigger to shoot all of these actions
are being taken just to maximize that
score and the agent doesn't know why
that action is good it only knows this
action will give me a higher score and
so the agent isn't able to really do
anything else besides that you can't ask
the agent to say hide behind one of the
pillars until that pillar is destroyed
or destroy all of the incoming space
invaders in one line and none of the
other space invaders so these are all
kinds of like different tasks that you
could give a human and they may
a little bit weird but humans would
understand what what it means to do this
and that's because humans have this
ability for mental simulation to imagine
what will happen if they take different
actions and so by giving our agents the
ability to imagine things and also plan
according to different tasks that it
might be given they're able to act more
flexibly and deal with these sort of
novel situations but humans aren't the
only form of intelligence we can draw
inspiration from we can also learn from
our cousins in the animal kingdom
let's bring in researcher Greg Wayne
within neuroscience
Greg's thing is memory and cognitive
architecture one of the things that is
quite clear is that animals have a
remarkable ability to deal with for
example very long time scales so
experiences that can be linked across
periods of time that is way beyond our
our current sets of agents the great
example I think is the scrub Jay the
Western scrub Jay they bury things they
prepare for the winter by scrounging up
a lot of food and putting it into
depositing different places hiding it
from each other and they love to steal
each other's food - they're scavengers
and they can remember thousands of sites
where they've they buried their food so
once all at once and they they can even
no detailed facts about it they know how
long ago they buried things that they
know if they were being washed while
they're burying things they know what
thing they buried there they have an
incredible memory for these events that
they have produced themselves how can
you tell that they know what they buried
because they have a preference you'll
see that they'll they like maggots more
than peanuts they'll go back to those
maggots first
having a kind of large database of
things that you've done and seen that
you can access and that you can use to
then guide your your goal-directed
behavior later you know I'm hungry mmm I
would love to have some maggots right
now where should I go find those that's
that's the kind of thing we would like
to replicate and there's another big
lesson we can learn from animals if you
want to teach dog to sit you don't write
a list of instructions move this muscle
bend your leg 45 degrees anything like
that instead you repeat the same task
over and over again offering punishments
and rewards as you go and if it's good
you give it a little bit of food that's
how we train dogs now I have a friend
who trains dogs to do things on iPads
using reinforcement learning so we've
already started on the path in AI of
merging reinforcement learning very
closely with how our AI is make
decisions and so on and that's how we
train them so you're essentially
training an artificial intelligence an
AI in the same way that you might train
a dog rewarding them for good behavior
ignoring that behavior very nice okay
how do you treat an AI what does it mean
to reward something that isn't
interested in doggy biscuits
here's dem assess Arbus well with
artificial systems or they really care
about is ones and zeros so you can
construct artificial reward mechanisms
for almost anything we've now moved away
from programming the system solution so
it now learns for itself so now going up
a meta level so now what we're really
programming or designing is reward
systems so it's kind of interesting that
that now is becoming the difficult part
is like how do you design curricula how
do you design breadcrumb trails or
rewards so that eventually they learn
the right things these systems but
there's also the idea of unsupervised
learning which is how do you learn
things if in the absence of any reward
and actually that's the issue with
reward learning in the real world as
humans or even as children there aren't
very many rewards it's quite sparse the
rewards even as a dog right the dog gets
a doggie biscuit every now and again but
has to decide every moment like what to
do and actually I think one of the
answer to that is what we call intrinsic
motivation which is internal drives that
have come through in animals have come
from evolution but we could also evolve
or build in those drives are very strong
and they guide the animal or the system
even in the absence of external rewards
so of course that might be things like
joy or fear or even things like hunger
these are all primal kind of internal
motivations that drive your behavior
even in the absence of any external
reward you're listening to deep mind the
podcast a window on AI research while
rewards might be a key part of how to
encourage AI to learn one of the main
aims of machine learning is for AI to be
able to teach itself to notice patterns
and shortcuts between tasks and make
themselves more efficient learners in an
ideal world engineers would like to
reach a point where AI can learn in a
similar way to humans picking up the
essentials of a new task in a matter of
minutes back to mat botnik an example
would be I went on holiday recently to
South America and I wanted to brush up
my Spanish and I knew exactly how to do
that I knew what resources were out
there for to begin with but more
importantly when I sat down to brush up
my Spanish I had a whole repertoire of
concepts that really guided me like I
know what it means to conjugate a verb
right I know that in certain languages
there are masculine to feminine forms so
this background knowledge helped me
learn much more rapidly than if I just
sort of was dumped into the middle of
you know a new language without
understanding what it means to learn a
language and we want systems we want
artificial systems that come armed with
these concepts it's not just about
language it could be video games we
could sit down in front of a new video
game that you've never played but if
you've played video games in the past
you kind of know how video games work
and that helps you to learn rapidly
the AI is what's known as narrow in its
focus now that might be diagnosing
cancer or playing video games but the
ultimate goal is to create something
much more powerful something called
artificial general intelligence with
precisely this ability of being able to
adapt to different situations to be able
to use the high-level concepts it's
learned in one environment and apply
them in another we don't want just a
system that's it's really good at one
thing we want a system that's really
good at lots of things but really what
we mean is we want a system that can
pick up new tasks that it's never
performed before we want an intelligence
where you say ok you've never solved
this kind of problem before but let me
let me tell you what I want you to think
about now and you could introduce them
to organic chemistry or something and
they would be able to work with that
humans can do this but getting machines
to do this is really really tricky and
it's not the only thing that we humans
can do that AI finds hard Gregg spends a
lot of his time trying to understand the
detailed mental processes behind
apparently simple human tasks you have
breakfast and you drink your orange
juice and you run out and then you think
to yourself god when I leave work I'm
gonna have to pick up some art shoes you
go through your workday you don't even
think about orange juice once and then
it springs to mind you know immediately
as you're leaving the office that you
need to go pick up some large juice when
you are going to buy the orange juice it
is actually of no value to your present
self the only self that will benefit
from buying the orange juice is yourself
at breakfast the next day so you're
actually you have to do something that
is incredibly prospective or thinking
forward thinking about the context of
your future self this is something I
mean people here really sit around and
sort of talk about and try and work out
what is it about your brain that reminds
you to buy orange juice at the right
moment yes because you can easily
construct virtual environments with
tasks for agents that we normally have
that have properties like this like
thinking minutes or hours ahead or
remember
something from hours ago that our normal
agents completely stumble on they cannot
do why is that they seem easy seems easy
to buy orange juice there's a theme
emerging here back in the 1980s hands
Moravec and his colleagues pointed out
that when it comes to artificial
intelligence everything is a little bit
upside down while the things that humans
find tough like maths and chess and data
crunching require very little
computation the things that we humans
manage without even thinking turn out to
be monumentally difficult for machines
it's a phenomenon that has become known
as more of X paradox like other
neuroscientists and psychologists here I
I find myself thinking about stuff that
seems really simple stuff that I do and
other people do really without thinking
about it and it just doesn't seem that
big a deal but it turns out to be those
some of those things turn out to be very
difficult to engineer into artificial
systems so picking things up putting
things down planning a route through a
building things that we can just do
without really much mental effort
sometimes proved to be quite difficult
to engineer an example of this just came
up as we walked into this room we all
realize that it was quite stuffy in here
and that we wanted to try to cool it
down so we all huddled around the
thermostat and tried to figure out how
to get it to do what we wanted and it
seemed to be resistant and at some
moment I thought wait a minute maybe
maybe the air conditionings just broken
and again that seems like a super simple
thing like you know what's such a big
deal about that but actually in AI
research we have a name for this which
is latent state inference we're trying
to infer some aspect of what's going on
which is latent or hidden and it turns
out in order to do that seemingly simple
thing you need a very rich model of the
world you need to understand air
conditioners and thermostats and what it
means to be broken and what's the
probability that it's broken and so
forth more of a paradox is often talked
about as some kind of profound mystery
it's used as evidence that while the
jobs of animal
some lawyers might be at risk in an age
of AI gardeners receptionist's and cooks
are secure in their careers for decades
to come
but deep mines founder demis hassabis
has quite a different take I think it's
quite obvious as a simple explanation
for it
when Marivic was doing AI the dominant
paradigm was expert systems so hand
crafting solutions directly to AI
problems think of it as building big
databases of rules of course if you're
going to do that that in itself is a
very explicit task programming that out
you know you have to know exactly what
you want to write and what rules you
want to incorporate and what that means
is the only tasks you can do that with
are the ones that you explicitly know
how to do as humans yourself and that's
things like the logical based like maths
and chess so weirdly the things that we
do intuitively ourselves and
effortlessly like walking and seeing and
you know all of these sort of sensory
motor skills seems effortless to us and
the reason is is because there's
actually huge amounts of brain
processing is going into that it's just
that it's subconscious its areas of the
brain we don't have conscious access to
we probably knew less about neuroscience
at the time so we didn't realize quite
how much processing goes on in the
visual cortex for example and so now we
know both of those things we know how
the brain works better and we've built
learning systems like alpha zero and
alphago so it turns out actually vision
is not any more difficult really than
playing go it's similar if you approach
it in the same way it's almost
impossible to reverse-engineer our
unconscious skills using the old methods
of handcrafted programming you have to
have a total and complete conscious
understanding of how something worked
before you could ask a computer to
replicate it but now the machines are
just beginning to mimic our subconscious
processes like vision and pattern
recognition there's no reason why more
of X paradox needs to necessarily be a
barrier in the future
[Music]
I have to be honest with you this single
idea more than any I've learned in
making this series is the one hit home
and underlines the power and potential
of AI for me all that we've managed so
far in everything that we've created
with machines are only the things that
we consciously know how to order them to
perform we're only just at the very
beginning of artificially mimicking our
subconscious processes - and that means
that there is an extremely exciting
journey ahead
but this partnership of studying
neuroscience and artificial intelligence
alongside one another doesn't just help
make our AI better
here's Matt Botvinnik and Jess Hamrick
again to explain we often talk here
about the virtuous cycle the opposite of
a vicious cycle right there's a virtuous
cycle between AI and neuroscience where
neuroscience
helps AI along and then AI like returns
the favor one of the reasons why we can
get this virtuous cycle between
neuroscience and cognitive science and
AI is because fundamentally we're all
trying to study the same thing which is
intelligence and so if we ask sort of
these more abstract questions about what
should an intelligence system do in this
situation we can ask that about humans
what would a person do in this situation
and try to come up with an answer we
could ask what should our a AI agent do
in this situation and try to come up
with an answer or if we have an answer
already in one of those fields we can
take the solution and apply it to one of
the other fields and I think that's sort
of really what enables this this ability
to transfer between the different fields
[Music]
this isn't just a theoretical flow of
ideas there are real examples of ideas
from artificial intelligence finding
their way back into neuroscience so
there's a neurotransmitter a chemical
that conveys messages in the brain
called dopamine in the 1990s people were
finding ways of tracking the release of
dopamine in the brain and very clear
patterns were being identified but
nobody really understood what they meant
why does the brain release dopamine in
this situation and not that situation
and as I understand the history some
papers hit the desk of some people who
were studying computational
reinforcement learning people like Peter
Dayan and Reed Montagu and they just saw
immediately that the the patterns of
activity that were being reported in
these neuroscience papers the dopamine
data could be explained by the math
that's involved in reinforcement
learning that has led to a real
revolution in the neuroscience of
learning
if you give a monkey a treat they get a
little heat of dopamine in their brains
it's the same in our brains too a little
burst of pleasure whenever something
good happens but in the 1990s
researchers realized that dopamine
wasn't actually the response to the
reward it was reporting back about the
difference between what the monkey
expected the reward to be and what it
actually received if you're walking down
the road and you unexpectedly find a 20
pound note it's much more exciting than
if you're collecting a 20 pound nose
that's owed to you by a friend and if a
monkey is expecting you to give it a
grape and you hand it a piece of
cucumber it's gonna be a lot less happy
than if you just surprised it with a bit
of cucumber from nowhere thing is AI
researchers were already using something
that acted in a very similar way in
their algorithms they'd get their agents
to make a prediction about what was
going to happen next and compare it to
what actually occurred but remember in
all of this the idea is to just take
inspiration from the way that our human
brains work not to make a
straightforward artificial copy because
our brains on exactly perfect
so we've heard how we can take
inspiration from the human brain from
the animal and even the bird brain to
create AI systems but this isn't just a
working theory anymore researchers
aren't just talking about what they want
to do they're also talking about what
they've actually managed to do let me
tease you with car I cover cholo
director of research at deep mind it's a
simple problem of course you can write a
program to solve that but the idea was
try to do deep reinforcement learning
try to come up with a system that we
think can generalize to different
problems more problems and once we saw
that it was a matter of weeks we had ten
or fifteen Atari games being solved if
you would like to find out more about
the link between AI and the brain or
explore the world of AI research beyond
deep mind you'll find plenty of useful
links in the show notes for each episode
and if there are stories or sources that
you think other listeners would find
helpful then let us know you can message
us on Twitter or email the team at
podcast that deepmind comm you can also
use that address to send us your
questions or feedback on the series
let's take a little breather see you
shortly


Go to Zero - DeepMind: The Podcast (S1, Ep2)

wherever you turn in this building
there are people playing of course
there's the usual pool tables and ping
pong tables but there are also go boards
chess boards vintage video game cabinets
and more recent strategy games like
Settlers of Catan because here you see
games are good I'm Hanna Frey I'm a
mathematician who specializes in
studying patterns in human behavior and
I'm also an author a broadcaster and
like rather a lot of us someone who is
fascinated by the potential for
artificial intelligence and the places
it can take us over the last 12 months
I've been working with deep mind the
london-based lab that has been called
the Apollo project of artificial
intelligence ready to go welcome to the
deep mind podcast open another door and
what do you know there's a chess
grandmaster pondering his latest move in
his early 20s Matthew Sadler was one of
Britain's all-time greatest chess
players he was there at the heart of it
all as a professional player when AI
started moving in on human chess
champions and if you know anything about
chess at all you'll remember this a
computer called deep blue has made chess
history by defeating the world's
champion Garry Kasparov this is the
moment in 1997 when
was defeated by machine it was the final
game of six and his opponent one of the
most powerful supercomputers in the
world seemed to have the upper hand the
game of chess supposedly a true test of
human intellect will never be the same
again the reigning chess champion Garry
Kasparov beaten in a devastating fashion
by a chess-playing computer built by IBM
known as deep blue one of those watching
was Matthew Sadler the big thing about
deep blue was that it wasn't actually
stronger than Kasparov won it won so
that was very annoying because everyone
you know was saying oh a computer's
beaten Kasparov you want to say yes but
it wasn't better you know Kasparov
didn't get self a bit psyched out and
you know deep blue paid well and and all
of that and then at some stage computers
came along that were a lot stronger and
then you could start using them and they
started showing you stuff that they
hadn't seen before you know incredibly
complicated tactical sequences where
you'd say that never works and yet it
does and then at that moment you sort of
give it up you sort of say okay they're
better than me and you start
appreciating what they bring to the game
and so is the story of deep blues
victory over Kasparov faded into AI
folklore people started to wonder what
game would be the next frontier for AI
research the most ambitious had their
eye on the ancient board game girl this
is not a game that responds to brute
force calculation
it requires intuition and an instinctive
appreciation of positions and beauty
unlike chess where by 2016 even a mobile
phone could play a credible game against
a grandmaster there was nothing that
came close to playing at the top level
of go but that didn't put off one man
from the challenge David Silva lead
researcher at deep mind I've always been
an ambitious person so I think at the
beginning of my PhD I set out on this
with this goal for myself to be able to
beat the world's strongest players
during my PhD which turned out to be a
little bit ambitious everyone was trying
to dissuade me from this course the head
of department took me aside and said
look you know you're just wasting your
time working on this project it's too
hard
no one will be able to do this he's seen
too many people banging the heads
against this problem and fail and he he
didn't want to see someone else in his
mind you know just bang their heads
against a problem that was too hard he
wanted you to be employable that's right
yes that's right go is an ancient
Chinese board game and although it's not
played much in the West it's arguably
the most popular board game in the world
it's considered one of the four ancient
scholarly skills of China and is taught
in school alongside sports or maths is
played on the pale wooden board engraved
with an elegant grid of 19 by 19 squares
the objective is very simple both
players are aiming to capture territory
on the board by enclosing it with their
pieces black or white known as stones
but the game itself is mind-bogglingly
sophisticated much much more than chess
even so David Silva was single-minded in
his belief that AI could master the game
go it was simply a question of how the
right approach it always seemed to me
was to allow machines to learn for
themselves this kind of intuition to
learn from themselves to be able to look
at a position and establish whether
black or white is ahead and this meant
machine learning in particular a method
with in machine learning called
reinforcement learning which is supposed
to be how humans and animals learn for
themselves through trial and error
experience but reinforcement learning on
its own wouldn't be enough it was only
when David began to work with the team
at deep mind that he spotted the missing
piece of the puzzle we'd seen a huge
revolution with something called deep
learning this is the ability for
machines to build up very rich deep
layered representations of knowledge for
themselves and that breakthrough seemed
to me to be the missing element that if
we could combine that process of being
able to build these very rich
representations of knowledge with the
kind of work which I'd been doing before
and reinforcement learning the ability
for machines to learn for themselves by
trial and error
if we put those two pieces together and
seem to me that that this was a recipe
that might have the legs to take us all
the way and by all the way David means
building an AI good enough to challenge
the very best go players in the world
this is a huge moment for both the world
of artificial intelligence and I think
the world of dope so far alphago has
beaten every challenge we've given it
but we won't know its true strength
until we play somebody who is at the top
of the world like Lisa Don
Sol March 2016 a decade after first wing
with the idea of designing a go-getting
machine
David's moment had come deep mines
contender the AI program alphago would
battle the eighteen time world champion
Lisa doll head-to-head in a televised
match of five games as the world's press
watched on with bated breath
I think the honest truth is that when I
flew out to Seoul in 2016 I was living
this very protected life as a researcher
working on this problem behind closed
doors just thinking about the
complexities of the problem and how to
how to make the system work and it was
only when I stepped off the plane and
walked into this hotel room most
absolutely jam-packed with reporters and
everything going on that suddenly the
penny dropped that this was a really big
deal that that actually you know the
consequences of this were far greater
reaching than had ever imagined there
were something like a hundred million
people watching the match as it
proceeded or something like 30,000
articles written about the match when
you arrived in Korea did you believe
that your your algorithm would win when
we arrived in Korea we actually got the
team together and I asked the team to
hold out a hand
we were playing five games and asked
everyone to hold up a hand to say how
many games after five we thought we
would win and you know many people make
many different predictions I actually
predicted for one David's prediction of
four one two alphago was a clear
expression of confidence but once the
match started the doubt started creeping
in I feel I made the mistake of under
estimating the quality of a real human
world champion when we were actually
playing the match I realized just how
immensely versatile Lisa doll was as a
player in his ability to push alphago to
his limits not just in one game but then
coming along again the next game and
trying a very different strategy and
in the next game trying a different
strategy like pushing them probing for
weaknesses all the way through and we
were pushing alphago in two regimes that
we'd never tested actually I don't know
if you've ever watched a televised game
of go but as a single stone is placed
calmly onto the board the commentators
and the audience watching on react with
the same ferocity of emotion and
excitement as they would a football game
even so there was one moment during the
first game where the response from the
crowd really stood out a moment where
even Lisa dolls expression fixed into a
look of shock his mouth fell open and
his hand came up to his face everyone's
expectation had been that Lisa doll
would eventually emerged triumphant it
was just a matter of time until alphago
made a mistake the game apparently to
the human commentators were still
balanced and at that point in time
alphago made an extremely bold move
quite nice to be here at David his
backstage with demis hassabis one of the
founders of deep mind watching alphago's
every move and their reaction is caught
on the camera in that is not a confident
face in Gautam's it invaded in something
which appeared to be least adult's
territory and alphago jumped right
inside to the region which seemed like
it belonged to Lisa doll and said okay
you know come and get me and it was an
audacious move and you could judge by
Lisa doles reaction that he he wasn't
expecting it he was expecting perhaps
more timid more computer-like response
and the reality was that alphago was
using its intuition to judge that if it
jumped in here it couldn't compute all
the way to the end all of these possible
outcomes but it had a sense that this
would work out well for it the first
round was a convincing victory for
alphago
roll on day 2 round 2 and alphago had
another surprise up its sleeve in the
second game the human commentators were
Ashley I mean the only way I can think
of his gobsmacked in their reaction
that's a very that's a very surprising
move this was the now-famous move 37
alphago had placed a stone on the fifth
line a move that no human player would
even consider there's these deeply
built-in beliefs about the game and one
of them is that in the game of Go you
can think of all these different lines
upon which stones could be played the
first line is closest to the edge the
second line the third line the fourth
line and there's a rule in the game of
Go which is that when you approach one
of these stones with diagonally it's
called a shoulder head that you never
ever do your shoulder head above the
fourth line and this has just been so
ingrained in Ingo knowledge because most
of the time it's true like most of the
time this is a very useful common-sense
piece of knowledge which helps go
players to exclude a vast range of very
bad moves but in this particular
position what alphago realized was that
playing on the fifth line and playing
the shoulder hit on the fifth line
actually just worked beautifully in the
context of this position with all of its
other stones in such a way that the
outcome was really favorable because in
the end of that game that stone ended up
being instrumental right it kind of
joined up to everything that's right
yeah that stone became so influential in
the game and it just worked forming this
big net that surrounded a vast swathe of
territory in the center
alphago just wasn't playing in a
mechanical way it was breaking the norms
and conventions of this ancient game it
was creating something a pattern of
playing that went way beyond the
approaches that humans had ever
considered and it was doing so
successfully move 37 would eventually
seal victory for the machine looking
back on the match Lisa Dahl spoke about
how this very move shifted his entire
view of the map
I thought alphago was based on
probability calculation and it was
merely a machine but when I saw this
move I changed my mind
shirley alphago was creative this move
was really creative and beautiful do you
think that was the AI illustrating real
creativity I think we need to challenge
ourselves to ask you know what is
creativity I think creativity should be
defined as anything which takes us out
of our expected patterns of behavior and
I think in in that sense it truly was
creative
alphago won the first three games but
the match wasn't over for Lisa doll just
yet in the fourth game he managed to
come back fighting against his opponent
Lisa doll was a true gentleman and we
couldn't have chosen anyone better to
represent humankind for this match he
not only strove his utmost to the very
end to play and devised all kinds of
amazing counter strategies to - alphago
but he dealt with the immense pressure
of having all of these people watch him
in really a profoundly human way he he
found it very hard he was humble
I think it hurt his pride to lose to the
computer but he came back and he found
new strength in that and was able to
ultimately emerge with immense pride at
having beaten alphago in one game and
been part of this pivotal moment for AI
at the end of the six days the final
score was alphago for
doll 1 so alphago became the first
computer champion at the game to go and
it was there a major result for
artificial intelligence and you won the
sweepstakes and I won the sweepstake
this is the deep mind podcast an
introduction to AI research from the
people behind
Alfred Go news of the March rippled
around the world Mac Botvinnik now deep
Minds director of neuroscience research
was one of the millions watching the
first reaction that people had in the
NGO community was oh it feels a little
sad that now there's a computer program
that can beat our hero but then it
didn't take long before people started
to realize wait a minute this is
actually really exciting we're not stuck
with our own limitations in terms of
seeing the possibilities of how to play
this game now there are new horizons
opened up to us we can find new forms of
beauty in this game I think that's sort
of in microcosm now what what I think we
can hope for from from AI more generally
and for David's this victory was always
the heart of a bigger picture it's not
really the case that I've ever had to
stop and question and say well what next
because the what next is clear we want
to take this further we want to build
machines which can achieve the same
level of performance but across all
kinds of challenging domains why stop
with go you once said the that you think
that the game of gos is the holy grail
of artificial intelligence do you still
think that as the case I think the
history of AI has been a number of
pivotal moments where for a period of
time a particular domain has been a
centerpiece of everyone's attention so
for a while the centerpiece of attention
was the game of chess when deep blue
defeated Garry Kasparov that marked the
end of an era when chess was no longer
their domain that people cared about and
and the world moved on but before the
world moved on from go David was curious
about just how far he could push the AI
really the open question for me was how
can a system learn for itself
entirely
with no human input if there was no
human supervisor there say here's the
inputs his the guidance here's the
examples of how humans play what if we
started really tabula rasa which means
start from a blank slate and the system
just has to learn everything for itself
starting from completely random play is
it able to learn for itself to play go
to the highest caliber of play that's
possible since their triumphs in 2016
David and his team have been busy
working on that new algorithm alpha zero
the original go beta alphago learned to
play by stunning millions of games
played by human experts alpha zero on
the other hand learns completely from
scratch from zero human knowledge
instead it picks up the game by playing
against itself millions of times
initially its gameplay as weak and
erratic but over time the system learns
to identify the best moves and
strategies it try something and if a
particular pattern is successful and it
ends up winning the game against itself
it uses that pattern more and if another
pattern ends up causing it to lose the
game it will play that pattern less and
over time it builds up this very rich
deep representation of knowledge one of
these so-called neural networks and it's
able to then go out and beat the world's
strongest programs which is better
alphago or alpha zero that go amazingly
we discovered that the system which had
learned completely for itself without a
single piece of human knowledge ended up
being far stronger in the long run and
defeated the original version of alphago
by hundred games to zero oh my god so
hang on you weakened it by giving it
human knowledge it turns out that we
have a tendency as human designers to
believe that we know how to make the
system stronger but quite often it turns
out that by putting our own
predispositions and preferences into our
programs we actually make them weaker
you
we're not needing any human input alpha
zero doesn't particularly care what game
is playing as long as you can give it
the rules is by no means limited to go
by now alpha0 has taught itself from
scratch how to master the Japanese game
of shogi it is currently the world's
best shogi playing machine despite there
only being one human in the deep mind
building who knows how to play and to
come full circle after only four hours
of playing itself alpha zero mastered
the game of chess to a superhuman level
[Music]
so far just a small handful of chess
greats have been able to test their
skills against the machine including
chess grandmaster Matthew Sadler who we
met earlier and Women's International
Master
Natasha Regan they've co-authored a book
about alpha zero called game changer and
outside of the research team at deep
mind they probably spent more time with
alpha zero than anyone here's Natasha I
played out four zero once and it wasn't
a very long game how many moves did you
get - oh I think it would have been less
than 20 and and I'd have to say it was
very direct
I played something sacrificial I thought
I'd try a opening thing it might not
know it and it exploited it very quickly
got its pieces out on attacking squares
straightaway and it won quite quick I
wouldn't get to 2020 mousse if I played
it take me down that's pretty yeah it
does have a very smooth human style
against me there was no need for it to
do anything complicated it just played
better moves over a long period of time
and just push me back gradually I
wouldn't have been able to guess that
was playing into the computer it would
if I had to guess a human it would've
been someone like Coulson or or Karpov
he's very smooth positional players who
just beat you by playing good moves
and that like alphago before it is a key
thing about the playing style of the AI
they're not like IBM's deep blue that
beat Garry Kasparov back in 1997 or any
of its descendants they playing with a
very mechanical style computer like
style they're very defensive they only
ever take calculated risks but alpha
zero on the other hand it conducts its
attack in quite a structured way so it
takes a count of the whole board and it
tends not to get attacked itself so it
gets to a position where its own
position is quite stable and safe and
then it can bring all its pieces in
consorted way into doing an attack it's
doing what humans do but only so much
better that's the thing it really
blending them that is it actually doing
original stuff yes it is
we've been playing chess for over 400
years so actually probably every single
move on the board has probably been
played once by someone somewhere you can
actually see my goodness some displayed
44 million games against itself it's
actually repeated our whole chess
history for itself and in that time is
just identified what the most important
things are of all that we've discovered
I mean that's what makes style alpha
zero has substance as well as style
right now in 2019 it simultaneously
holds the titles are being the best
player in the world at go shogi and
chess and it's not just being greedy the
whole point of this project was to build
an intelligent system flexible enough to
respond to a range of problems and while
alpha zero might not quite be able to
tackle cancer diagnosis or energy
efficiency there is a good reason why
deep minder playing with building these
all-purpose machines
in the world of games our goal of course
is not just to play chess or go or or
whatever our goal is to have impact on
some of the world's most challenging
problems which are facing society but in
order to do so we need to gain
understanding we need to really deeply
understand these systems for ourselves
first and games provide the perfect
testbed for doing so games are like the
ultimate mini universe you know all the
rules there's a clear winner at the end
you can look back at the end of the game
and decide what went wrong and if you
lose well it doesn't matter you can just
start another round the big problems
though the ones we eventually want to
tackle they are quite a lot more
complicated the real worlds are really
messy place and you end up with these
amazingly complex things like human
beings and how they interact with each
other and societies and companies and
all these amazing things which we've
built up in our world we need to be able
to understand them to be able to have a
hope to apply something like alphago to
to make progress in order to make
progress then we need to be able to
apply systems that can operate even when
the what rules are unknown and that is a
big remaining challenge which is not yet
being addressed by alphago or alpha zero
if you want to know more about how games
have been and continue to be used as a
testbed in AI research then head over to
the show notes where you can also
explore the world of AI research beyond
deep mind and we'd welcome your feedback
or your questions on any aspects of
artificial intelligence that we're
covering in this series so if you want
to join in the discussion or point us to
stories or resources that you think
other listeners would find helpful then
please let us know you can message us on
Twitter or you can email us podcast at
deep mind calm
[Music]