Out of the lab (incl. AlphaFold) - DeepMind: The Podcast (S1, Ep5)

[Music]
this is deepmind
the podcast and I'm Hannah fry over the
last year I have been getting an inside
look at current research into artificial
intelligence or AI and we've been
talking to scientists researchers and
engineers about how things stand and
where we headed tracing the fast-moving
story of one of the biggest challenges
in science today so if you want to be
inspired on your own AI journey then
you've come to the right place
up until now in this series we have
largely looked at what AI is capable of
in the lab or in the world of games but
the ambition of people working in AI is
to help solve problems in the real world
right now people are trying to use AI to
help with everything from predicting
traffic jams to monitoring endangered
species and here at deepmind they've
also been working on a few things
now we could call them case studies but
where would the fun be in that so I'll
tell you what let's one of those big
trailer things shall we in this episode
we learn how a I might potentially help
save the sight of thousands of eye
disease sufferers we get the output from
the first neural network and we can
interrogate that as doctors and making
decisions for our patients help break
down the Enigma of protein folding we
can learn what the processes or a
description of the process so that we
can take a sequence and then predict its
structure an impact on our growing
energy demands if that waste he isn't
taken care of somehow it will literally
melt your laptop
first I want you to meet Sims with a
spoon Sims is originally from South
Carolina she's now the program manager
at deepmind and she is a big believer in
the potential benefits that AI can bring
when people think about deep mine if
they think about games they're largely
thinking about the research side but
trying to solve for intelligence is
literally to use that intelligence to
make the world a better place the world
has no shortage of problems we need to
solve we've got some really big ones
some smaller ones but for the areas of
immense complexity things where there's
lots of data and lots of permutations or
a huge combinatorial space of
possibilities sometimes that's really
daunting for human brains to try to
figure out stuff in the real others just
really complicated yeah it's really
complicated and you know real-world data
is really messy but if we can use AI to
try to find a path through that
complexity then we can solve our
problems faster than we could if we were
trying to do it on our own nowhere is
that more true than in medicine in 2016
deepmind partnered up with Moorfields
Eye Hospital NHS Foundation Trust in
London to try and apply deep learning to
AI scans with the number of people
suffering from sight loss in the UK set
to double by 2050 the trial had the
potential to be hugely impactful peers
keen is a consultant ophthalmologist at
Moorfields Eye Hospital and National
Institute for Health Research and knows
firsthand about the demands placed on
doctors one of the huge problems that we
have in ophthalmology not just in the UK
but all around the world is the huge
number of patients that we have to deal
with so in particular and the National
Health Service we get nearly 10 million
clinic appointments in the UK every year
and that's actually 10% of all clinic
appointments across the whole NHS and
it's a number that's increased by more
than 1/3 in the past 5 years why is it
increase so I think it's increased
because of the ageing population
I think it's increased because certain
diseases like diabetes are on the rise
and so we just have to deal with a lot
of eye problems related to that so for
example I can tell you about just one
condition which is age related macular
degeneration or AMD and AMD is the
commonest cause of blindness in the UK
comas cause of blindness in Europe and
and in North America as well and for
just that one condition nearly 200
people developed the blinding forms of
AMD every single day just in the UK and
so the challenge that we have is that
those people have to be seen and treated
in an urgent fashion the problem then is
that in for example in 2016
Moorfields Eye Hospital in London where
I work received 7,000 urgent referrals
from the community as possible wet AMD
the blinding form of this condition but
after 7,000 referrals only 800 patients
actually had the severe form of the
disease with so many people getting
urgent referrals is inevitable that
patients are gonna have to wait weeks to
be seen by a specialist and during that
time you have thousands of people who
think they've got an eye disease that
will threaten their sight who don't
actually need to worry and hundreds of
people with a curable condition whose
sight could be saved but is slipping
away while they wait peers told me about
one of his patients Alain Mona Alain
lost her sight from macular degeneration
in her left eye completely more than 10
years ago before there was good
treatment and in 2013 she started to
develop blurring a vision in her good
eye she went to her High Street
optometrist who looked into her eye and
said I think you're developing AMD in
your good eye you need to be seen and
treated urgently because now we have
good treatments for this she got an
appointment to another hospital in the
NHS and it was six weeks later so can
you imagine if you were at home and
you're losing your sight and your good
eye and you're told you have to wait six
weeks when there is a treatment that's
available and if that was my family
member
I'm treated in six days not in six weeks
here is where the AI comes in and the
idea is simple patients who are referred
to more fields will already have had
pictures taken of the back of their eye
by their doctor or the optometrist in
their high street opticians these are
two and three-dimensional images known
as Oct scans that can show up any one of
50 different diseases if you can use
artificial intelligence to filter
through those images first and triage
the patients you can flag the people
with a serious disease get them in front
of a doctor sooner give them an earlier
diagnosis earlier treatment and
potentially save their site before
artificial intelligence and before the
sick sent recent successes of deep
learning the traditional approach to
programming an algorithm to recognize a
photo of a cat for example would be you
would write all that code to describe
the features of a cat cat has whiskers
cat as a tail then you'd say some cats
don't have a tail some cats don't have
fur etc etc and you try and write
thousands or hundreds of thousands of
lines of code to describe that with deep
learning we don't do that with deep
learning we show many examples often
thousands or hundreds of thousands of
pictures of cats - a neural network and
it will extract the features of interest
itself and learn how to recognize a cat
well we simply do the same thing but
with eye diseases the Machine doesn't
care whether it's looking at cats or the
backs of eyes yeah to a large extent yes
and I think the reason why this
collaboration has been successful so far
is because Moorfields Eye Hospital is
one of the oldest one of the largest AI
hospitals in the world and we have huge
numbers of O CT scans to train these
neural networks does it work then I
think the results are amazing I think
they're they're jaw-dropping I think
that the algorithm we've created is on a
par with world-leading experts at
Moorfields
in triaging these are CT scans
but there is quite a big difference
between spotting cats in pictures and
picking out eye disease for something so
important how do you know the algorithm
is getting it right
how can a consultant be sure to see what
the algorithm saw or feel confident to
overrule it if they don't agree
well the key has told me is about
building an AI that doesn't just tell
you what it's found but also shows you
and to do that the AI needs not one
neural network but two so the first
neural network is trained to identify
all the disease features on the scan and
the second neural network is trained to
take those disease features and to use
them to make a diagnosis on the skin so
it's first going through and kind of
highlighting areas that don't look
totally normal yeah anything that looks
suspicious yeah the second ones coming
in explaining what's going on and all of
those and using that to come to a final
decision exactly yeah and you can see
all those areas that that first neural
network has highlighted yes so that's
one of the great advantages of this
approach we get the output from the
first neural network and we can
interrogate that as doctors making
decisions for our patients so if you've
got bleeding in the retina or if you've
got leakage of fluid and water logging
of the retina it will highlight all of
those features so if you see that a
patient has diabetic eye disease then
you can see the very typical features
that have led it to make that decision
which gives a lot of I think reassurance
for healthcare professionals who would
be using this there's a double whammy
with this approach not only can it
reassure the consultants helping them
with the diagnosis they're already doing
but there is hope that the AI might one
day also be able to advance our
understanding of the eye itself in 2018
another group of researchers decided to
see if they could use deep learning on
images of the retina to predict the sex
of the patient now the best an eye
doctor could manage would be a 50-50
guess
but to their astonishment the algorithm
got 97% right
no ophthalmologist in the world has any
idea what it is that this algorithm is
picking up on in a photograph or any
theory as to why the male and female I
might be structurally different but the
AI has found something that they're now
trying to understand and in the more
field study even when the algorithm gets
the diagnosis wrong it might still be
picking up on something that the
professionals hadn't spotted what was
interesting was that when we looked at
the cases that the algorithm got wrong
we actually had to take a step back
because it seemed like some of those
cases were very ambiguous challenging
cases where maybe the algorithm had made
the right answer and our gold standard
was at least open to debate
really so really kind of like jaw
dropping there at the results that we
were getting jaw dropping indeed so
that's AI dipping its toe into the world
of medicine but how about one of the
most fundamental problems in science
when I spoke to a very senior researcher
about what he thought were the most
significant problems in biology he his
top problem was understanding the brain
and how that works
his second problem that he thought was
the most important was understanding how
proteins fold this is sandy Nelson a
product manager for deep mind science
program and as sandy told me it's hard
to overstate the importance of proteins
it is the most cited topic in 50 million
scientific papers so many of the terms
are used to query thinking about medical
conditions are actually underlying
proteins so we think about the immune
system how that works of all that's
proteins we think about hormones we know
that regulates so many functions in our
body so of course drugs are a lot to do
about small molecules interacting with
proteins but there's many other ways
which proteins are important for
thinking about say disease
so we know for example Alzheimer's and
some of those neurodegenerative diseases
are to do proteins or proteins are
implied
proteins are the building blocks of all
living systems stretched out straights
they're just big long chains of amino
acids a bit like a ribbon but they fold
in on themselves and make these giant
three-dimensional structures stuck
together with peptide bonds now the
number of different ways a protein could
fold is vast think origami here except
mind-bogglingly complicated human
microbiological origami with 10 to the
power of 300 possibilities and
scientists care a great deal about
exactly what shape those final folded
proteins end up as part of the reason
the proteins are so useful for taking
part in so by cat many biochemical
processes because they are specific they
can target very very specific points in
some process that specificity comes from
the uniqueness of their shape so when we
think about proteins are the the go-to
molecule for anything you need to do in
a living animal if you want to try and
understand why some of the proteins have
gone wrong or to create some kind of
intervention understanding that process
of creating structure from sequence is a
first step and maybe designing proteins
or understanding why it might go wrong
the function of the protein whether it
is to detect light in the eye or fight
disease or speed up reaction rates is
determined by its unique
three-dimensional structure the question
is how does the protein go from one
state to the other from the ribbon to
the final folded structure what is the
objective heaven is it that in the end
you want to create something where I
tell you a sequence of amino acids and
you tell me what the structure will look
like so it is simplest level yes if you
could do it as accurately as it can be
done in a lab that says huge amount of
effort in theory you can just observe
the shape of the final folded structure
the most common way of doing this is by
bombarding crystals of the protein with
x-rays
inferring its shape from the way that
these beams are scattered but that is
hard to do it can cost hundreds of
thousands of dollars for each protein
structure and take months or even years
of work it's so hard in fact that max
Peretz won a Nobel Prize in 1962 just
for figuring it out for one single
protein hemoglobin there is an
alternative though the final structure
of the protein is actually determined by
the chain of component parts the forces
and charges that are acting on each of
those individual amino acids so in
theory you could use the physics to
predict how the protein ribbon is going
to fold but it's going to take a lot of
number crunching so if you had a
gigantic enough computers you know
supercomputer level I could give you a
string of amino acids and you could tell
me what shape it would end up as but the
problem is that we just don't have the
computing power to crunch through it not
at the level of modeling all the forces
so we can explain why the protein folds
the way it does using our understanding
of chemistry and physics but because of
the size and the complexity of the
molecules there are so many forces we
can't model everything and here's where
AI comes into it we think that there's a
another level of abstraction where we we
think we can maybe find a summary
description of all those forces and
that's again too hard to come at through
analysis but maybe we can learn that
because we've got a huge data set which
says well this sequence folds this way
and we know that that's reliably the
case so using machine learning maybe we
can learn what the processes or a
description of the process so that we
can take a sequence and then predict its
structure
here is a problem with a very clear
objective correctly predict how a chain
of amino acids is going to fold and a
vast vast number of possible ways to get
there ai is perfectly placed to cut
through that complexity the only problem
is that even with AI on your side these
things are so enormous ly complicated
that you still can't cut a clear path to
predicting how a protein might fold
based only on the physics thankfully
though there is a trick that you can use
to simplify the problem and give your AI
a head start the facts that proteins are
so diverse can help you constrain the
problem although I should warn you as a
mathematician I found this stuff pretty
hard to get my head around so I'm going
to try and walk you through it nice and
slowly proteins like organisms have a
long evolutionary history they can
sometimes be small random mutations in
the string of amino acids every now and
then a mutant protein will differ from
its normal version on just one of its
corners where if you unraveled it back
into the ribbon of amino acids the
markers of that mutation would show up
in more than one spot you can imagine
this as though you've got your folded
ribbon scrunched up in some complicated
shape in your hand and then you take a
felt-tip pen to one corner of it now if
you unfolded your ribbon and flattened
it out you would see that the pen would
have stained various spots along its
length so working backwards then if you
start with a flattened ribbon and notice
something strange in a few different
places marks that hint a consistent
mutation you know that however the
protein ends up being folded you found a
big clue those stains must have to be
next to each other in the final protein
collect up all of those clues and you
have greatly simplified your problem is
it like you've got this this vow
just sort of landscape of options and
you're trying to build walls to pen
yourself in yes that's exactly right
because these proteins are so large they
could fold in so many different shapes
so we need to find clues that allow us
to eliminate whole mass of shapes so we
can concentrate just on a much smaller
number you're making the problem smaller
that's right you're listening to a
podcast from the people at deep mind now
every two years there is a big protein
folding competition called CASP critical
assessment of structure prediction
competition over the course of three
months academics from around the world
compete to predict the structures of
amino acids using algorithms the
structures of these particular amino
acids have already been confirmed
through traditional observation so it's
possible to judge who comes closest and
in 2018 deepmind entered it's AI program
alphago and we had to look at how other
people did protein folding and we saw
how they used evolutionary information
and what other people had been doing was
they'd been looking at a sort of binary
constraint we said these two amino acids
should be in contact shouldn't be in
contact
whereas what deepmind did is we looked
at the probability of different
distances between those amino acids so
that's really like just saying well we
tried to retain some more information or
learn a better function for describing
that relationship between proximity of
amino acids so in terms of the fences
that you're building on your landscape
I'm not going quite far with this
knowledge you were making sure that you
weren't throwing away any information
that's why our fences were more subtly
defined or a bit more clearly delineated
or they were less fence like and more
like just a sort of that's right nice
but that actually ended up making the
prediction more accurate yes that's
right so it's it's a very very
complicated function but we
were able to learn that and so once we
were able to learn to kind of
essentially retain that extra
information that's one of the key things
that made our system more successful
[Music]
with the problem reduced the a I could
get to work doing what it does best for
three nail-biting months the deep mind
alpha fall team worked on the
competition
turning sequences of amino acids into
predictions of three-dimensional folded
shapes we didn't have any strong signal
which told us how well we're performing
we could see that we were not perfect in
many cases so it is very hard to find
out how well we were doing compared to
other people there are so many fantastic
researchers that were publishing great
results until we actually went through
that sort of organized assessment very
very hard to really figure out how well
we were doing
and then finally it was the moment of
truth of the 43 strings of amino acids
they were given the team came closest to
correctly predicting the structure for
25 of them the team that came in second
only managed three a staggering result
by anyone's standards you're kind of
downplaying this because I was talking
to a few academics when this this result
came out and of all of the results that
have come out of deepmind this is the
one that's got the scientific community
most excited yes and and that's because
this is a classical scientific domain
it's a grand challenge in science that
many people have worked on so it's
something that many many scientists care
about deeply they can see what the
potential impact is and it's been known
to be a very hard problem so we've been
able to make a step change on a hard
problem that's been worked on for over
50 years in terms of interventions then
is this just something that biologists
and scientists will get very excited
about in terms of kind of blue sky
research understanding protein or is it
something that could end up having an
impact in real people's lives so I guess
it's similar to all sort of biomedical
research it's fundamental so it has huge
leverage of essentially will affect many
many things but it needs to be
translated into something specific for
it to have immediate impact and on
people's lives so for example if we
think about the drug discovery process
part of that process goes on in labs and
it's very abstract and all to do with
chemistry but ultimately that process
does produce medicines that we can buy
or prescribe which ultimately will
affect our lives so this is at the start
early part of that process for example
although the long-term implications of
protein folding have the potential to
impact all of us it's not exactly a
topic that most of us are coming
face-to-face with on a daily basis but
one issue that we are all facing is that
of climate change
you remember Simms who met earlier while
her team decided to focus their efforts
on one specific climate challenge energy
consumption in data storage and they
started its by looking for a place where
burning far more energy than we need to
because it turns out your emails are one
of the things that are warming the
planet if you think about the things
that we all do online every day whether
that's sending an email doing a Google
search looking at dog videos on YouTube
you know the number one videos cat
videos for me it's dog videos I make I
make nice-nice all of that requires
compute power and you know the
information that we you know data we
send data that we store when information
is disseminated all of that runs through
a physical space ie the data center it
and it takes a lot of energy to do all
of those actions that we rely on on the
internet because there I mean there are
actual sort of physical warehouses that
are holding all of those cat videos o as
there yes they are physical spaces and
if you think about the amount of energy
they consume a data center you know in a
large industrial kind of setting can
consume the same amount of energy as a
small town I mean these things are
massive and they require a lot of energy
to run they also require a lot of energy
to cool all those emails sitting in your
Inbox the four dog videos you're
streaming simultaneously the request you
sent to the server to download this very
podcast every one of those things
requires computing power in a data
center somewhere collectively
data centers now use 3% of the world's
energy the equivalent of a whole new
country that just popped up on the map a
few years ago and all that computing
generates heat lots and lots of heat if
you imagine how hot your laptop
gets when you're streaming Netflix or
you know the for video is online imagine
that but multiply it times a million
what if that waste heat isn't taken care
of somehow it will literally melt your
laptop or in the case of the data center
it will melt your server that's why your
laptop has a fan
that's why data centers have cooling
that needs to happen there we have to
keep them at a temperature so they don't
melt and you and I can get our dog
videos off YouTube and I guess just
cooling down those data centers takes up
at a vast amount of energy yes it does
we are you know we're talking about
chillers required that are the size of
buses in order to to keep them cool and
this is where a eye comes in so imagine
you were trying to control the cooling
of a data center and a human being who
you know it's usually a facility manager
data center operator just has to kind of
dials to control and that was all you
had to do to control the entire Center
now that is a vast oversimplification
like a fine and a key yeah exactly
just those tips you could figure out the
the best you know is it just aircon is
it just fan is it both is it neither
like that's not that many options right
you could figure that out but when it
turns into a huge number of pieces of
equipment with set points on every
single one which are all things you can
change by some degree that then interact
all the sudden you've got a vast number
literally a number of options that is in
the billions and that's just too much
for a facility manager a data set
operator a human being to try to control
so this is where we think AI is it's the
perfect space for AI because AI can
adjust a vast amount of information more
than the human brain can and can help us
figure out which of those permutations
which of those combinations actually is
the optimal path forward
how does AI cut through one of this this
complexity
we can ask a model to figure out okay we
want to keep the data center at a
certain temperature but we want to use
less energy to do that here all of the
ways that you can manipulate the system
please figure it out the setting might
look quite different to a game of chess
or go but the principle ideas here are
exactly the same again you have a very
clear objective namely keep the center
cool while using as little energy as
possible and a vast vast number of
possibilities of how to get there that
the AI has to find a path through and
once it does the AI tells you how all of
the dials should be set across the
across the center exactly what's that
points to change and by how much to
change them and does it work it does
work that's the best part about it we
saw that with direct AI control ie you
know getting those recommendations and
having a I feed them directly back into
the physical infrastructure of the data
center going through lots of safety
constraints we saw a 30% reduction in
the amount of energy required to cool
Google data centers which is math which
is massive and a really exciting number
I don't have to confess I've seen the
graph yeah of what happened when you put
the AI in indirect control of all of the
dolls and it is staggering I mean you've
got this it's sort of bumpy line that
goes along about how much energy is
being used and then it's all it looks
like those graphs were the pound crashes
yeah she's in terrible news it drops off
a cliff and then you kind of have it
bumping along the bottom of the graph at
which point you take the AI away from
being in control and it says switch back
over to human control and just jump
straight back up to where it was before
it's amazing is the AI running the
cooling system in the day centers right
now yes it is which is fantastic and
hoping to roll them out to even more in
the future now that we've proved that
this works and works well with more data
with more practice in other words the AI
gets better over time so 30% is a
fantastic number but it's increasing you
know a rules and heuristics don't get
better over time but AI does that's the
best part about these systems
and that is why there is so much
excitement about a eyes potential here
at deep mind and AI labs around the
world if you would like to find out more
about applying AI to energy healthcare
and scientific problems or explore the
world of AI research beyond deep mind
you'll find plenty of useful links in
the show notes for each episode and if
there are stories or sources that you
think other listeners would find helpful
then let us know you can message us on
Twitter or email the team at podcast at
deep mind calm you can also use that
address to send us your questions or
feedback on the series now so we have
another break


AI for everyone - DeepMind: The Podcast (S1, Ep6)

welcome back to the sixth episode of
deep mind the podcast my name is Hanna
Frey I am a mathematician who's worked
with data and algorithms for the last
decade or so and I spent the last year
at deep mind an organization that is
trying to solve intelligence and then
use it to solve some of society's
problems there are an awful lot of
people working in the field of
artificial intelligence moving forward
our understanding of the whole area and
for many of them is a terrifically
exciting place to be with breaking new
frontiers of problem solving and seeing
great leaps ahead but before any of that
hits the outside world the first
Inklings of me breakthroughs here at
deep mind come in the form of regular
poster sessions have a Marla can carry
out the following simple tasks where I'm
gonna give you a number and a secret
symbol and what the sum between those
two are and you have to infer from that
what the values but it requires our
agents to have some properties that we
think are desirable like learning to
learn having a memory and processing
those memories
we're studying analogical reasoning and
logical reasoning is very important
because it's key to scientific discovery
also human reasoning the main question
we ask is how can we design your own
networks that are able to do our logic
to be single my poster is about
verification of neural networks in this
day and age when we deploy in your
networks into the real-world
applications we want to make sure that
these newer networks are safe for
example if you have an image classifier
we don't ever want to predict the cat to
be like a car with something like that I
always say deep mind is a bit like
academia on steroids okay it's still
like an emu but we have a lot of compute
a lot of great people clustered together
a lot of help to like manage ourselves
so while there is obvious excitement
about AI research this new era of
artificial intelligence also comes with
concerns there is unease about the way
it might be implemented used and abused
for the rest of this episode we are
looking at the more human side of
technology and the fight to find a
future of AI that works for everyone in
2017 deepmind set up dedicated teams
working on how AI impacts ethics and
society with the aim of making sure that
the algorithms designed in this building
a positive force for good bang on I know
you're thinking surely algorithms aren't
ever good or bad in and of themselves
it's how they used that matters after
all GPS was invented to launch nuclear
missiles and now helps to deliver pizzas
and speakers playing pop music on repeat
have been deployed as a torture device
isn't the technology itself just neutral
good question and I think something a
lot of people say and believe and I can
see why they say that this is Verity
Harding co-lead of deep mind ethics and
society I hit let's earth famous saying
about as long as there's been fire
there's been arson but you can use
something that's for good you can use it
for bad but I think actually as we're
developing increasingly so
stated technologies that have real
impact on people's lives it's not really
an acceptable thing to say you can't be
building something that's going to have
this kind of monumental impact or
potentially transformative impact and
not care about how it's going to be used
it's that part of a concern then the
technology that might have been built
for one purpose ends up being used in a
different way I think definitely that's
some of it I think definitely that's
some of it because you could foresee a
situation where you're building a facial
recognition tool because you want to
allow somebody to quickly find pictures
of their husband or wife or mom or dad
and that's a great thing
but that facial recognition technology
once developed could of course be used
to target political dissidents and pick
them out of a crowd you know and so I
think that is definitely one of the
concerns that you might create something
for one purpose and it be used for
another on the topic of facial
recognition Brad Smith the president of
microsoft recently refused a request by
a US police department to install their
algorithm in cop cars and body cameras
and he's publicly called for more
careful thought and societal dialogue
about potentially regulating the use of
the technology and here at deep mind
more generally there is a strong sense
that the people behind the science have
a duty to investigate the wider and
perhaps less predictable impacts of
their work I don't think it's okay to
build something whether that be a
product or a service and put it out
there and and just just hope that you
make the world a better place I think
it's important that you are deliberate
and intentional about why you're
building this who are you building it
for what are you hoping to do what is
your intention with this technology and
if you start from that premise then I
think you're more likely to get to a
better outcome where you do the good
that you hoped you were going to do the
problem is that without these steps it
is very easy for unintentional
consequences to creep up on you you only
need to look at the news stories about
social media from the past
two years to see just how much
algorithms have changed our society in
unexpected ways Lila Abraham is deepmind
CEO and has over 20 years experience
working in the tech sector she has seen
firsthand how hard a booming industry
has found it to keep up with being
responsible in 2006 I went into the
middle of the Amazon and we built a
computer lab and health care so we put
in Internet and computers etc we knew we
had a responsibility not to just leave
it there but to Train folks to take care
of it to think about the sustainability
but I think that's kind of where things
tend to end epochs means something very
different now and responsibility means
something very different now because
technology is in everybody's hands it's
no longer limited to a few people for a
specific application it's a lot easier
to get into the tech sector and to make
technology that can have value to people
and at the same time that comes a lot of
responsibility that I don't think in
general the tech sector has taken into
account but the last few years have
shown how hugely transformative and
disruptive AI can be and brought sharply
into focus the very possible negative
outcomes of ill thought through
technology
but as Verity told me the tide is slowly
beginning to turn much of the drive for
a conversation about ethics is coming
from within the technology community
itself three years ago in 2016
some of the scientists from different
labs at different companies met at a
conference and we're talking about how
excited they were about the potential
for AI to do a lot of good but
acknowledging that a technology that's
so powerful that it has the potential to
be transformative in a very good way
must also have the potential to be
transformative in in not-so-good away
and so they came together to say well
what can we do about it and so the
partnership on AI was born it includes
members from Amnesty International
Electronic Frontier Foundation the BBC
and print
ten university amongst many many others
and together they're hoping to come up
with best practices in AI making sure
that society stays firmly at the
forefront of engineers minds so the
partnership on AI interestingly was
founded by the biggest tech companies so
it was founded by deep mind but also
Google Facebook Amazon IBM and Apple one
thing that's really interesting about
the partnership in AI is that the board
membership is made up of independent
board members and representatives of the
company and so it's creating a space
where those different groups aren't
siloed from each other having debates in
different rooms and not listening but
somewhere where honest people with the
best of intentions can come together and
challenge each other and scrutinize each
other and hold each other accountable
but also have Frank open honest debate
about issues where reasonable people can
disagree I really believe that the
outcome of that will be better
decision-making both in companies but
but elsewhere as well because it
sometimes get quite heated in those
conversations you know my experience of
it is that it doesn't get heated it but
it's passionate so people aren't angry
with each other and there's there's not
aggressive argument but people are very
honest I'm very open and very
challenging but that's been received
really well in in all cases how do you
protect against rogue companies just who
are not a part of these groups just
doing whatever they want if enough
companies and enough groups sign up to
something and it becomes the norm its
then really obvious when people aren't
doing it and I do think people being
kind of cooled out for that it'll no
longer be tenable to not operate in the
way that everybody else is operating
it's not just theoretical concerns about
runaway applications of AI that's
prompting these conversations but real
examples of algorithms that have already
been let loose on the world with real
question marks about whether their
benefits outweigh their harm a notorious
example is the use of AI in the criminal
justice system now you may have heard of
these algorithms already
when a defendant appears in court the AI
can assess a defendant's chances of
going on to commit another crime in
future and that risk score is then used
by a judge to help decide whether the
defendant should be awarded bail and in
some cases how long someone's sentence
should be there is good justification
for something like this because there is
an enormous amount of luck involved in
the human judicial system studies have
shown that if you take the same case to
a different judge you will often get a
different response if you take the same
case to the same judge on a different
day you'll often get a different
response judges don't like giving the
same response too many times in a row
and so if a series of successful cases
of bail hearings have gone before you
your chances of being successful fall
and there is even evidence to suggest
that judges tend to be a lot stricter in
towns where the local sports team has
lost recently using AI to help make
these decisions can help to eliminate a
lot of that inconsistency but you have
to tread pretty carefully if you without
thought and care and due attention to
the history of racial prejudice in the
criminal justice system build something
that claims to be able to predict
somebody's likelihood of reform and
rehabilitation and reoffending then it
is likely at least in my view that
that's going to fail if you build
something with the intention of
addressing those biases and you work to
include the community in some way you
could there potentially be a beneficial
outcome maybe but I haven't seen it yet
and by fail you're really talking about
treating black defendants differently to
white defendants absolutely and once you
tend to look at the algorithms and the
data that they've been they've been
built on oftentimes you can see whether
they were built on data that was already
biased so of course this was the outcome
the issue came to public attention in
2016 after a group of US investigative
journalist from ProPublica published a
damning report of one particular
company's criminal risk scores their
study showed that the algorithm was
twice as likely to wrongly categorize
black defendants as being likely to
reoffend than white defendants now I
should just point out that deepmind does
not build these systems but the whole
industry alongside the partnership in AI
has been part of the conversation of how
to address them one of those people is
William Isaac a social scientist at deep
mind he says that the 2016 Pro Publica
investigation made people realize that
switching over to algorithms doesn't
make decisions any more objective even
with AI and ML tools you are getting
into the social environment where you
actually have the same norm the same
kind of like systemic biases they're
still all present so it's really hard to
say that somehow this will replace all
of the kind of subjective preconceived
notions about certain groups or
historical biases against them and that
you can start all over again and so I
think that was the wake-up call was that
is not as objective as it seems and that
as a result we still have to grapple
with those questions the problem is the
data which gives the algorithm
predictive abilities a questions like
how many times were you arrested as a
juvenile but if you are say a young
black man in America it doesn't matter
how Laura biting you are the chances are
that you will have had many more
negative interactions with the police
then someone exactly like you who
happened to be white and if you're using
that data to dictate who deserves to be
given bail or not then you
and serious risk of perpetuating
societal imbalances going forwards this
is Silvia Chiappa a staff research
scientist at deep mind research don't
fully understand what this furnace is
about they also look like a messy area
in the sense that involves it is not
purely technical problem and it's very
difficult to understand what how to
define fairness and it's difficult to
separate the technical part from the
ethical one this is an important point
because defining exactly what you mean
by fair is surprisingly tricky
of course you'd want an algorithm that
makes equally accurate predictions for
black and white defendants the algorithm
should also be equally good at picking
out the defendants who are likely to
reoffend whatever racial group they
belong to and as ProPublica pointed out
the algorithm should make the same kind
of mistakes at the same rate for
everyone regardless of race ethically
you'd want all of those things to be
true but technically that's not always
going to be possible if your data set
has bias in it there are some kinds of
fairness that are mathematically
incompatible with others and even if you
could guarantee all of these things
there are still a number of ethical
issues to contend with how do you
measure fairness who is excluded from
your definition how do you make those
decisions transparent and ultimately how
do people contest the decisions made by
those algorithms see I told you it was
tricky coming at this from two very
different perspectives William and
Silvia started looking into the bigger
issue of fairness in algorithms even
though we had kind of different
frameworks me as a social scientists and
Silvia is a machine learning researcher
the actual overlap between how we would
approach this and basically the
assumptions that are embedded within it
were remarkably similar and actually
part of us was saying like oh like look
at these papers and social science that
are kind of making the same point they
just hadn't actually had a way to
actually communicate that formally
you're listening to a podcast from the
people at deep mind in April 2009 teen
William and Sylvia Co published a paper
on fairness in AI entitled a causal
Bayesian networks viewpoint on fairness
in it they show that no matter how fair
algorithms might be if the data they're
learning from is biased we still can't
trust their results I don't think it is
possible to find technical solutions
that are completely satisfactory at some
point we need to take decisions whether
the kind of unfairness is acceptable or
not but we can't advance a lot and
that's why we need more researchers
involved and not just machine learning
researcher person researcher from
different community to be raising
awareness about this this problem find
solutions but as we will never be able
to find completely satisfactory a
solution from a technical viewpoint at
that point we need to take decision this
is important to talk about these such
that we are something that is missing at
the moment I do think this is
fundamentally like a societal ethical
question and challenge and it will
require lots of stakeholders to address
if you have let's say data set a facial
recognition tool that's designed to find
missing children
what threshold do you set as a society
where you say ok this is acceptable we
maybe are less successful at identifying
children with darker faces
what threshold do we say that's
acceptable because that's not a
technical question that's a social and
political question and a normative
question even if you do have a
classifier or facial recognition
software that's fair the application of
it may be in unfair ways and so that
might present a second question that is
separate from the actual kind of like if
you decide on the threshold right if
you're just using it in neighborhoods
that are predominantly one group or one
ethnicity that presents a whole nother
set of challenges for whether or not
that's an ethical use of a particular
technology
you can't assess whether these
algorithms are good or bad in isolation
they don't exist on there
you have to place them in the context of
the world that they're being used like
the criminal justice system or in health
care
here's Verity Harding again this is what
I mean by it being a kind of much bigger
discussion that potentially the use of
algorithms is highlighting my fear is
that people will kind of get a checkmark
that says we've tested and this
algorithm isn't biased and therefore you
should feel free to use it and that to
me isn't going far enough I think there
needs to be a further discussion then
about but is this making those decisions
that were already bad worse or more
quickly and therefore more of them and
you know that that kind of thing but
things are changing
he's William on what has happened since
that ProPublica story broke they're
going back and reconsidering what
measures they collect right and going
back and trying to create more robust
data sets thinking about who is
collecting the actual data itself will
it be ever perfect where we have bias
free purely pure data no I don't think
that that's that something is ever gonna
happen but I do think that people will
be skeptical when people ask about what
datasets are used and they don't get a
satisfactory answer right I do think
people will ask is this data set
representative does it have balance
across different groups so people will
start asking the right questions and
interrogating datasets and in models
more aggressively and I think that will
lead to better outcomes and crucially
more people are now being included as
part of the conversation in the
aftermath of some of my work and among
many others on predictive policing many
cities in California actually started
implementing citizen boards so when
police departments wanted to acquire a
new police technology that included uses
of machine learning or artificial
intelligence that they had to go in
front of a citizen board and actually
have the the local community evaluate
the tool for different metrics including
fairness and bias
[Music]
getting different voices involved in the
conversation is essential to making sure
that we build a future that belongs to
all of us because what seems obvious to
one person just wouldn't occur to
another your perspective is hard-coded
into the work that you create there are
clear examples of this everywhere
outside of AI able-bodied people
designing buildings that disabled people
can't use or new tights and plasters
that only work if your skin is one
particular color presumably the same as
the designers and the algorithms that
we've created they're really
highlighting this issue like the ones
used to automatically screen CVS and
predict which candidates will fit best
in a company here's Verity Harding again
if it's based on historically
discriminatory hiring decisions by
either intentionally or unintentionally
bias humans then it's going to kind of
recreate those those patterns like if
you've got a company where white men
have succeeded yeah and they're looking
for candidates he'll succeed it's going
to pick out white male series yes
exactly and if the people building the
technology are all white males as well
then the likelihood of paying attention
to that potential bias and being aware
of area we all have our blind spots then
the likelihood increases we've seen
driverless cars that don't spot
pedestrians with darker skin tones tumor
screening algorithms that aren't as
effective patients with ethnicities
other than white European and lots and
lots and lots of issues around gender
all of this is kind of inevitable unless
you have a range of different viewpoints
in your design process the most
important thing in my point of view for
ensuring that these things are if not
not biased but that you're being
intentional about what you're building
and aware of the potential bias is that
your team is a diverse team is that you
have a broad set of voices involved and
it's actually much simpler to do that
than
it's suggested and the issue of gender
diversity has been a particular focus of
late I think there's plenty of young
women and girls who are really excited
by science and STEM subjects and it's an
easy get out to say that there aren't
enough women in stem and that's why
workforces aren't diverse but actually
it's much more about making sure that
it's a safe space for women and girls to
work that they're not discriminated
against once they're there that you're
able to not just attract and hire them
but that you're able to keep them and
make sure that it's a place where they
feel comfortable working and so I think
it's much more important that we look at
how women are treated in science than
just dismiss it as something that girls
aren't interested in a young age Linna
abraham deepmind CEO is very conscious
that diversity is still a problem in the
tech sector as a whole talk about things
that keep me up at night right so here I
am a professional of 25 plus years with
an engineering background a mom also
raising nine-year-old twin daughters I
would have hoped by now we would have
solved the problem and yet we haven't
we're like at the same numbers are flat
but they're all steps being taken to
address it there's the short term stuff
you can do which are things like you
diversify your candidate pools you if
you're doing university recruiting you
look at a broad range of universities
and ones that have that have a broader
student representation and have support
structures often in place to help the
students through their academic and
communities you look at job descriptions
and ensure that you don't have
unconscious bias reflected and your job
descriptions you so once you're in the
recruiting pipeline then you may need to
make sure candidates have the right
experience we are being very deliberate
about how we invest back in education
ai is something that will change future
generations so how do we make this a
field that's more accessible so for
example whether its funding diversity
scholars and universities or funding AI
chairs and universities to
to increase the pipeline and I think
that helps fuel some of the academic
aspects as well as support our like
long-term recruiting this isn't just
tokenism that we're talking about here
this is about making better technology
diversity and diverse perspectives will
create a GI faster and safer and with
just a better a better result because
one of the things I worry about is how
do we avoid our own internal bias a lot
of the work around deep reinforcement
learning started from specific pockets
and many people grew up in those labs or
those universities and you know we they
brought their former colleagues and so
we have a pretty strong network of
people that have known each other for a
long time
which is fantastic and they can really
advance certain aspects of our of our
work and yet there are other areas that
are emerging how do you teach curiosity
how do you how do we ensure that we
minimize bias and the code that we're
writing who's to say what intelligence
is and isn't unless you have a better
representation from society and that's
just on the research side on the
operation side to you think about things
like okay think about public policy like
if you're asking governments to think
about how they're going to treat
artificial intelligence then you want
people that are representative of the
the constituents of the population if we
want to be focused on education making
sure that we're not just focused on the
specific schools but our broader range
so we're bringing more people into this
space I just think it's going to be
imperative for us to truly solve
intelligence that we're just going to
need to have more diversity so he's
positive that well solution is probably
a bit grand their words to suggest but
it is part of the way forward making
ethics a kind of keystone every stage of
the process rather than having is an
afterthought oh absolutely we have to be
thinking about our responsibility for
the technology we develop and to
candidly to
as a whole every step along the way and
I think that there's something quite
special about being Pittock ordered out
of London versus being based out of
Silicon Valley
I'd love Silicon Valley it's where my
career has has really developed and yet
you're surrounded by technologists you
know from the billboard signs to the
marketing and promotion here in London
it's so multicultural and I feel like
it's part of your daily life you need to
be thinking about the work that you're
doing and how it is going to impact all
the people around you but of course we
can't just leave the solution solely in
the hands of the people who are
designing these things it's our future
to the public and government should also
have a hand in this my impression is
that people want to understand what
they're using and want to understand
what makes it work and how it works but
they more importantly want their
representatives and the people tasked
with keeping them safe and secure to
understand it too and I think that's why
we've seen a bit of a breakdown in
recent times if you want to know more
about ethics diversity and fairness then
head over to the show notes where you
can also explore the world of AI
research beyond deep mind and we'd
welcome your feedback or your questions
on any aspects of artificial
intelligence that we're covering in this
series so if you want in the discussion
or point us to stories or resources that
you think other listeners would find
helpful then please let us know you can
message us on Twitter or you can email
us podcast at deep mind calm
you
[Music]


Towards the future - DeepMind: The Podcast (S1, Ep7)

welcome to deep mind the podcast where
we're exploring the world of artificial
intelligence we're assessing what we
know and what we don't know we're
looking at what we're trying to know
we're calculating what we will know
mapping out where we're going and
working out how we will know when we get
there I'm Hanna Frey and I'm an
associate professor in mathematics and I
spent the last 12 months at deepmind
and in this episode we are going to
start looking forwards to the future the
self schooled variety of AI known as
artificial general intelligence but
first you know what I don't feel like we
really given you the Grand Tour of this
place deep mind headquarters in Kings
Cross in London so follow me in my
squeakiest of sneakers closed doors
coming through so the language they do
machine learning understanding language
with me is the extremely likeable Coria
Caracciolo one of the right firsts
deepmind is joining way back in 2012 is
actually it's a group of people who are
either focused on agents better memory
more planning how can we get him get
those into the agents all of your rooms
are named after famous mathematicians
Gauss room Hedy Lamarr Oh actually I
know her I know her she's actually a
very famous actress you know I liked her
best in I take this woman with Spencer
Tracy but she was actually also
scientists right the deep learning group
is more interested in just like coming
up with algorithms and architectures on
any data domain some seriously delicious
equations on the whiteboard I've learned
yes
I mean blackboards and whiteboards if
you have an idea in the corridor it's
important use it this area is the
machine learning so one of the main
projects going on here with this group
is the imitation just just demo so
Sabres in the office this is the area
for the neuroscience group they do a lot
of thinking about what are the important
problems what are the challenges that
the discipline is really hard yeah table
football seven of my course a regress ah
the reinforcement learning team this is
one area that has been core to what we
try to achieve with a GI if you're
interested in agents it has to be active
they've got some proper weight eat
expose that's not not exactly bedtime
raging is a molecular electronic
structure theory I know what all of
those words mean but just not in that
particular order sweeping funds sleeping
I didn't know that they've been coming
in for a year no one told me about these
there anyone in there
look in my defense she didn't have a Do
Not Disturb sign on the door okay enough
of that already back to the simple stuff
like solving the Enigma of intelligence
just one tiny snag before we get there
first we're gonna need a proper
definition of what intelligence is
because intelligence you see it's quite
a slippery beast to pin down
[Music]
we've got a bit of a head start when it
comes to human intelligence although it
might not be everyone's favorite metric
IQ is one of the most stable
psychological tests we have and it does
a pretty good job of measuring limited
intelligence markers like reasoning and
logic but IQ still doesn't get us any
closer to a definition of intelligence
if we're going to get anywhere with this
we need to define it properly we need
some way to capture what we mean by
intelligence that works just as well for
humans and dogs as rabbits and machines
and there have been a few suggestions
for what intelligence is over the years
in 1921 the psychologist V Henman said
intelligence was the capacity for
knowledge and knowledge possessed which
sounds quite good on the surface until
you realize that it also applies to
libraries libraries can possess
knowledge do libraries count as
intelligent probably not in 1985 Marvin
Minsky the cognitive scientist said that
he thought intelligence was the ability
to solve hard problems and that seems a
bit more like it it also captures what a
eye has already proved it can do
here's Riya Hansel senior research
scientist at deep mind in the last few
years there's a huge number of different
narrow specific things that programs can
do as well as a human they can interpret
your voice as well as a human they can
maybe translate from English to French
and back again almost as well as a human
they can recognize things and images
almost as well as a human these sort of
narrow specific things
that word narrow is not to downplay the
transformative power of this sort of
thing just in this series we've looked
at energy conservation medical diagnosis
and protein folding all of which
certainly show the machines ability to
solve hard problems and all of which are
examples of narrow AI but real
intelligence general intelligence that
needs something else something more some
scientists have described intelligence
as the capacity to learn or to profit by
experience others think it's about
adapting and thriving in the environment
you find yourself in but whoever you are
smart disagree that's somewhere along
the line intelligence is something about
your ability to interact with an
external environment and being able to
adapt has to be part of it too so you
can't be fully familiar with the
environment you've got to be able to
deal with unanticipated challenges that
get thrown at you if you're intelligent
in 2007 after going through hundreds of
competing arguments Shayne leg one of
the three co-founders of deep mind wrote
an influential paper in which he and his
co-author tried to pin down precisely
what was meant by intelligence and here
is the definition that they came up with
intelligence measures an agent's ability
to achieve goals in a wide range of
environments and that is what they are
aiming for in this building here's a
reminder of what a senior research
scientist Maurice Shanahan told us in
Episode four the holy grail of AI
research is to build artificial general
intelligence so to build AI that is as
good at doing an enormous variety of
tasks as we humans are so we are not
specialists in that kind of way you know
a young adult human can learn to do a
huge number of things and can indeed do
an enormous number of things and can
adapt to a huge number of different
challenges you can learn to make food
you can learn to make a company you can
learn to build things to fix things you
can do so many things
to have conversations to rear children
so although those things and we really
want to be able to build AI that has the
same level of generality as that and
that's really still an open challenge we
don't really know quite how to get there
but if anyone has an idea of what it
will take its dem assess our best the
CEO and co-founder of deepmind
will be talking to him in the next
episode of our Adi but for now here is a
little glimpse into his thinking I'm
waiting to see a lot of key moments for
example I think a really big moment will
be when an AI system comes up with a new
scientific discovery that's of Nobel
prize-winning level that to me would be
a big watershed moment so you know
capable some kind of true creativity in
some sense I think other big points will
be when it can use language and converse
with us in a naturalistic way it's
capable of learning abstract concepts
these are all things that I think a
high-level cognitive abilities that
we're nowhere near yet and I think will
be big so I'm posting on the way
now it's reasonable to be asking
yourself how do you even approach such a
colossal task where do you even start do
you totally and completely believe that
AGI as possible yes back to my tour
guide core I cover to Luke he's the
director of research at deep mind I
believe that they will come where we
will be at that stage right now we are
not right now all we can do is go back
from that and then have a hypotheses
about the important problems the
important algorithms that we need to do
the important solutions like those key
things we need to have and then start
building more and more and more so let's
take an example then let's say the first
day that someone at deepmind decided
they wanted to look at the problem of
navigation so building an agent that you
can drop into an environment yes
where it's going do you just have like a
big brainstorm on a whiteboard of all of
the different possible aspects that
might contribute to being able to build
that agent yeah it starts with that
because if someone must work on
something like that then probably there
will be a good number of people here who
would be interested in the same thing we
will start discussing okay what is the
goal like when we say navigation you
gave a particularly good example right
like going from here to a given location
how are we going to specify it what kind
of environment this is what kind of
control space does the agent have all
these start affecting what kind of
algorithms we should use and are we
going to do this purely from vision are
we going to do this in a simple
environment in a grid world or we're
going to start from a grid world and
then we need to think about the path
towards going to a 3d environment are we
thinking about actually also putting
this on a robot like with real vision
like all that discussion starts
happening there's massive then I mean
just that one form is enormous it is but
that's why it's also research a big part
of it is trying to constrain the problem
space like being able to write down and
specify what what you want to do and
then making sure that it is actually a
challenging problem and there are good
metrics that we can quantify or we're
saying you're successful right exactly
and that itself as I said is quite an
iterative process and getting that
critique getting that view from other
researchers starts at that point in time
because like when the research is at the
idea space at the initial stages it's
actually quite important to formulate
the right problems and and sort of the
right context Adi is not going to happen
overnight it's the reason why as you
heard on the tour there are so many
different research groups in this
building you're not going to crack Adi
by attacking on only one front a
formulating the right problems means
going beyond individual skills like
navigation and drilling down into the
building blocks of intelligence and
- from the beginning of deepmind if the
goal is about a GI then it has to invoke
control it has to involve an active
algorithm and that is why you need to do
reinforcement learning that is why you
need to work on agents we need agents
that can interact with their environment
that can learn through trial and error
what actions to take
what karai calls its policy it's a
fundamental thing that you would expect
to find in any intelligent being humans
dogs or agents and so if the scientists
can get it right in one application the
lessons learned should apply elsewhere
the gist of training an agent is you can
start completely from scratch and then
slowly the agent builds that knowledge
that strategy of how to achieve a
certain task and it creates it builds it
comes up with its own strategies it
comes up with its own policy so then of
course looking at it and trying to
understand does that pose make sense
does that buy is it doing that policy
sometimes it's so surprising because
it's something that we haven't thought
about before sometimes you look at it
and you see that need that doesn't make
sense yes it it achieves something but
it's clearly not optimal can you
remember when you first realized or
believed that AGI was possible so we
tried these agents on Atari games and
they didn't do much and we couldn't we
couldn't make them work right like there
was a there was a team of like like like
several people there and we couldn't
make them work and then slowly we
started simplifying the problem and
simplifying the problems and find the
problem and we ended up with a very tiny
simple trivial problem like really just
like a five pixel by 10 pixel image and
one pixel moving in that image and the
agent trying to control that and once we
reduced the problem to that which I call
the emne stuff reinforcement learning
really
we could find the working solution we
started having this deep reinforcement
learning agents working right because
it's a simple problem of course you can
write a program to solve that but the
idea was like try to do deep
reinforcement learning and try to solve
it that way try to solve it from pixels
try to come up with a system that we
think can generalize to more to two
different problems more problems and
once we saw that actually it was a
matter of weeks we had ten or fifteen
Atari games like being sold like from
that tiny thing in the matter of weeks
you go to Atari that that was a big
moment that's what we keep in going that
like we select more and more diverse set
of problems that we think are important
at the end for AGI that is a key point
here intelligence is an agents ability
to achieve goals in a wide range of
environments so to get to AGI we need
agents to solve harder and more diverse
problems
you're listening to deepmind the podcast
a window on AI research the closer we
get to AGI the more powerful and
sophisticated this technology gets and
the more we rely on it in our everyday
lives the more dramatic the consequences
could be of us misunderstanding the
limitations of the algorithms and that
is why in parallel to pushing the
science forward researchers are also
working to ensure right now the agents
are reliable adaptable and crucially not
corruptible
[Music]
you show the image of a bus to a neural
network and it will say this is a bus
right there's a bus in this image well
it turns out that you can take the same
image modify it a little bit which is
almost invisible to a human eye but the
neural network will say that it's
actually an ostrich this adversarial
attacks are things some most of the time
that are invisible to the human eye that
doesn't change the actual content of the
image too much we don't perceive it but
because these are algorithms and bastes
they're they are sensitive to even very
small fluctuations in the input data
then it changes it changes the output
why does that matter why do you want to
stop that from happening in the real
world ai well for two reasons one of
them as I said is robustness because
when we trained these algorithms we want
them to be useful in real world and we
train them on datasets that we captured
from real world
but like you cannot know exactly what's
going to happen
of course what kind of data is going to
be at the end of today this algorithm is
going to consume so we want to make sure
that the algorithms are robust to these
kinds of potential like noise like
things that you want your algorithm to
be robust to that right and from another
point of view it's about safety being
able to say that like someone maybe with
an adversarial intent won't be able to
change the output of this algorithm
output of this neural network just by
making very small adjustments to the
inputs we have a we have a whole
research group actually on that on
working on rigorous and more robust
artificial intelligence because yes
these are real things in the end we are
training these algorithms and as I said
it's not just like looking at them but
we're trying to do more quantifiable
research on understanding why they are
doing what they are doing and trying to
interpret that and part of it is also
understanding how robust they are
[Music]
we have already seen real examples of
just how fragile intelligence can be
researchers have shown that you can add
a little bit of black tape to a stop
sign that will trick a driverless car
into speeding up the tape is so subtle
that it would look innocuous to a human
driver but it's just enough to make the
algorithms inside the car miss read it
as a 45 mile an hour speed limit sign
instead of an instruction to stop other
scientists have worked out how to fool
facial recognition algorithms into
thinking someone is mere Yakov it just
by making them wear a specially designed
pair of tortoiseshell glasses and the
images that are used for medical
diagnosis can end up giving wildly
inaccurate results just if a slightly
different brand of scanner was used to
take them you want to reach a state
where you can actually guarantee that
things like that won't happen that's the
main idea behind doing this research so
not sort of try to defend against
particular adversarial attacks or
examples but actually come up with
systems that are going to be robust no
matter what from the other point of view
of course if you are thinking about
agents there's the whole safety issue
what do you mean when you guys talk
about safety what are you actually
talking about if we have an intelligent
algorithm making decisions for itself
you want to have some sort of guarantees
that I mean it's acting with its own
policy it is aligned with what you
intended it to do if you think that
these algorithms continued learning all
the time
then we want that process to be also
producing an agent that is aligned with
what we have intended for it to do fill
in the gaps for me never give an example
of what you're trying to avoid it
becomes quite technical in the sense
that it's not like you are trying to
avoid the agent from making a mistake
right because mistakes happen right like
betraying policies and they are not
always optimal data
to the writing all the time it's not
about that really it's more about you
have a learning algorithm and you want
to make sure that it sort of conforms to
certain boundaries of behavior in
essence if we can develop safe robust
ethical AGI then the impact could be
staggering
but as demmas mentioned so too could the
discoveries on the way to Adi has agent
learned to solve increasingly hard
problems Trevor back is a product
manager for the deep mind science
program he played a key role in the
Moorfields Eye Hospital collaboration
that we talked about in episode 5 but he
also has a hand in deciding what the
future holds for deep mind and what
challenges and opportunities lay in
store for them to tackle next so we're
really at the stage of exploring what
other areas we should work on but the
possibilities are endless if you look at
the way the alpha fold system works
there's nothing really specific in there
to protein folding it's around
understanding the way that atoms
interact the way that you can build
material from base concepts so looking
at material design is a really exciting
stage you know could you design or
imagine a high-temperature
superconductor that's been sort of
brought to life through an AI algorithm
right is there more efficient ways of
looking for those types of materials why
do we care about high-temperature
superconductors so this is the amazing
opportunity of working in in science so
our previous work in healthcare has been
focused on very specific problems and it
takes a lot of time and effort and
energy to build an AI system that works
for those specific problems if instead
you can spend your time and energy
solving a fundamental science question
then perhaps you can you know instigate
a whole new field of interest and
potentially impact a much wider array of
problems so why is superconductivity a
problem you know if you could solve
superconductivity not only could you you
know solve a lot of the energy problems
by having a larger
genetic field around fusion but you
could also create a new type of
computing system you know there's lots
of opportunities that come from just a
single breakthrough in one of these
areas I think it's important not to
understate this because the idea that
you could have some kind of an impact on
nuclear fusion
for example the implications for the
earth and humanity are just enormous
right exactly I think this is the reason
I came to D mind was the opportunity to
have the greatest impact I could in the
world and I think AI is one of those
revolutionary technologies that
amazingly could you know impact
everything we do but also everything we
think about doing in the future and the
sort of opportunity to explore and
search the space of opportunities is
really something that's well designed
for AI to do it's a very efficient
search algorithm and so if you're able
to set up the problem in such a way that
is searchable via a are you really could
have 10 a hundredfold type of increase
in the opportunity for finding novel
materials or finding anomalies in
astronomical data you know finding new
types of stars finding more black holes
you know all these types of
opportunities are available simply via
the application of AI and it's not
stupid to say that actually lots of the
biggest problems that face humanity at
the moment are science problems right
like you know access to food and water
climate change healthcare all of these
things is stuff that AI can make
progress in I think that's right I think
a lot of the the sort of physical
problems that exist in the real world
are certainly things that if you can
make a difference to some of the the
sort of foundational science aspects you
know you could reduce the cost of energy
essentially down to zero or you could
make food more readily available across
the world and so that would really help
society progress in a number of ways
it's a tantalizing prospect solving
intelligence and creating a G I will
take the full range of research explored
in this podcast and more memory
reasoning logic learning language
embodied cognition and more so much more
and we're going to explore some of those
ideas in our next episode when I meet
with deepmind co-founder demis hassabis
he tells us how he created the world's
leading AI research outfit reveals what
keeps him up at night and opens up about
his hopes for the future I'm just
fascinated and also troubled by the
things around us that we seemingly don't
understand all the big questions you
know the meaning of life had the
universe start what is consciousness all
these questions which I feel like a
blaring sort of klaxon in my mind that I
would like to understand and my attempt
at doing that is to build AI first if
you would like to find out more about
artificial general intelligence or
explore the world of AI research beyond
deep mind you'll find plenty of useful
links in the show notes for each episode
and if there are stories or a sources
that you think other listeners would
find helpful then let us know you can
message us on Twitter or email the team
at pod cast that deep mind calm you can
also use that address to send us your
questions or feedback from the series
you