Me, myself and AI - DeepMind: The Podcast (S2, Ep7)

hello and welcome back to deepmind the
podcast
this episode is all about how ai is
already having an impact on the world
around us
shall we begin
[Music]
uh excuse me what are you doing starting
without me
i'm the real hannah fry
i'm only trying to help i heard you were
unavailable to present this episode so i
offered to step in
unavailable i'll take it from here thank
you very much
that voice you just heard there it was
generated using wavenet
voice synthesis technology trained on
audio recordings of my voice
in this episode we're going to be taking
a look at some of the ways that
deepmind's technology is already being
used out in the real world
including how wavenet can recreate the
voices of people with vocal impairments
it was really touching to see his family
and him listen to the voice his family
cried
because it's something that's so
personal
how neural networks can help anticipate
natural disasters
it is important to know if there's going
to be a buildup of a catastrophic storm
that's going to create flooding
and how ai could even transform the game
of football
so a coach might say to the system what
will happen if i move fabinho from
defense to midfield
welcome to episode 7 of the deepmind
podcast
me myself and ai
let's go back to that snippet of audio
from the beginning of this episode
generated by wavenet the deepmind
podcast aficionados among you may
remember that wavenet doesn't just
generate speech
it can also compose music
and we used a little bit of it in our
first series
but when it comes to creating human
sounding voices wavenet has improved
considerably over the past few years the
motivation however has stayed the same
everything from reading documents out
loud for the visually impaired to making
your smart speaker sound more natural
here's how zachary gleicher a product
manager on deepmind's applied team put
it
text-to-speech research has been
happening for decades and everyone knows
that texas speech voices have
historically sounded pretty robotic a
classic texas speech voice is the
stephen hawking voice british people
describe its accent as american here he
is speaking on bbc radio 4's desert
island discs in 1999 but the americans
say it is scandinavian
it's not because
people want robotic voices it's because
it's an extremely challenging problem
humans have evolved to be able to
understand very subtle nuances in how
things are set and if there's one little
thing that sounds off then people are
like oh that sounds robotic like if we
were to create a
dog barking generator
people would be like oh my god that
sounds just like a dog and you wouldn't
be able to perceive
any of the differences because our
brains not trained to know what good dog
barking sounds like meanwhile your dog's
in the corner being like it's so fake
exactly
[Music]
before wavenet the general method for
generating speech was called
concatenative text-to-speech
you'd get someone in a recording studio
and you'd record hours and hours
trying to capture all the phonemes in
the alphabet so that you have a real
diverse recording set
in production you stitch together the
voice recordings so imagine you wanted
to say the cat sat on the mat
and you had a recording of someone
saying the word the and you had a
recording of someone saying cat you
could stitch those two words together
but the problem there is that the voice
is going to sound like that
instead of stitching different bits of
pre-recorded words and syllables
together
wavenet directly models the raw waveform
of the voice building up less than a
millisecond of audio at the time
first it will scan the text you give it
for abbreviations and convert them to
something that can be fed into the
speech generator
like changing
hwy 101 to highway 101
[Music]
the second step is to try and predict
the intonation of how something should
be said based on the text around it
the can be read as the the
the
depending on where in a sentence it
falls
each would sound wrong if it was used in
the wrong context
now the third and final part is the
acoustic modeling
acoustic modeling focuses on who it
sounds like
if i pretend to sound like my brother on
the phone it still sounds like me my
friend will be able to tell it's me if i
say a sentence with a different tone of
voice you still know it's my voice
back when deepmind launched wavenet in
2016 you needed about four hours worth
of audio samples from a person to model
how their voice sounds
but now you can do it with just a few
minutes worth of audio
one of the big breakthroughs was a
process called fine tuning which makes
it possible to co-train voices together
google has built an enormous data set
with professional voice actors reading
out the same text
the model learns from all of these
samples how particular words are
pronounced
each new voice that is added to the
database results in an improvement to
all of the other voices and all that's
then needed is a small sample of a new
voice to provide the finishing touches
if you like that make the voice unique
to that person
that's why we call it fine tuning
because it's a way to just kind of
fine-tune the model based off that one
additional speaker because the
difference between your voice and my
voice for instance even though you're
speaking a different accent as a male
voice actually the way that we roll from
one word to another will have lots of
similarities yeah of course
after getting the all-important consent
from the person whose voice you're
creating it's as simple as recording
around 10 minutes of high quality audio
and matching those up with written
transcripts for the model to be trained
on
given that you had high fidelity good
recordings of yourself because you are a
podcast creator we were able to do that
without having to send you back into the
recording studio i had a hannah fry
voice bot where i could type anything
the power the power
hi there i'm a mathematician author and
podcaster who's fascinated by artificial
intelligence and i'm the real doctor
hannah fry
it's really good i know it's good what's
really awkward is that it's picked up on
a couple of the indonesians that i know
i must make but i've never really
noticed that i'd make yeah like did you
hear it went fascinating
[Laughter]
i love it see it knows you better than
you know yourself oh it's so cringe god
do i sound like that there were two
words that i thought sounded a bit off
one was mathematician
there was like a very hard bit in the
middle which i think isn't how i would
say the word i'm a mathematician author
and podcaster and i'm the real doctor
hannah fry
the second thing was how i said my name
it's a bit like you know when someone
reads your phone number back to you but
rather than saying like oh seven eight
one three they're like oh seven eight
one three video and then you're like
yeah there's something's gone wrong in
my mind yeah yeah if you synthesize long
sentences you'll notice there are some
things where you'll be like oh that
sounded a little weird texas speech
isn't solved yet we've reached the point
where voices sound perfectly natural in
many instances
and the challenge now is largely about
how natural it is given a certain
context
so for example if i wanted to text a
speech system that would
say
oh hannah i really like your sweater
now you say say it sarcastically hannah
wait oh
that's where we're really lacking is
like how do you capture everything
when i'm not trying to crack one of the
unsolved millennium prize problems you
can find me chillaxing with a cup of tea
and that quintessentially british tea
time snack a scone
am i that breathy bloody hell
sound like i'm on a sex line
the other thing that's worth saying
actually about that the way i'm having a
conversation with you now where i'm a
little bit more up and down and a bit
more energetic say
is a different sort of voice the audio
that this was trained on was the script
that i read out for series one so
inevitably then it will end up being in
that style exactly people will be like
make the voice sound happier make the
voice sound sadder and that's really
hard if you don't have examples because
the model has to learn what happy hannah
sounds like eventually though could you
have a system where the ai understands
how a happy voice differs from one
that's reading a podcast script
and can make those changes appropriately
yeah you can make hannah sound generally
happier but for people who know you
really well it's just like wait it
sounds slightly off because you might
have certain quirks about your voice
that can only be learned if you hear how
you say something you know if you always
elongate a certain word when you're
happy
as people do just so happy yeah yeah i
don't know if that's how you say things
when you're excited fabulous
i don't know everyone has their quirks
this next recording hints at how this
could be used dangerously you could
use the text-to-speech synthesizer to
say anything
hello i'm dr hannah frye and i'm here to
tell you that ufos are real when i went
into my garden yesterday i noticed these
strange dark circles on my lawn
how can you make sure that it's not used
for nefarious purposes
we thought a lot about this technology
on how it could be abused i think like
the thing that we care most about
is that people's voices are not created
without their consent
that's why we have not open sourced the
models we haven't made the data sets
available to mitigate a lot of those
risks but also there's a lot of cool
mitigations i think one that excites me
is that you need a script to be able to
create a voice
and
you could have that script be
you saying that i give consent for my
voice to be created
there is some research that's being done
that watermarks audio
is the idea that in creating this
artificial voice
you deliberately imprint tiny audio
signatures that you could see with a
certain piece of software perhaps but
that are
inaudible to the human ear so then you
can go in and say ah look this one is
fake exactly but here's the thing
watermarks could be removed people
might not consider that it's fake and
there are a lot of companies who are
releasing this technology it's not like
deepmind has the secret sauce there's no
surefire silver bullet way to stop this
technology being used by harm there's
ways to mitigate it but the same way
that we don't trust photos today
everyone sees a photo and be like is
that photoshopped i think it's going to
be the same with audio for better for
worst i think people are going to just
not trust what people are saying within
an audio recording and it's unfortunate
to see people
using it irresponsibly
because it might spoil a lot of use
cases that are really helpful for
society
zachary told me about a partnership
between deepmind and google called
project euphonia in which wavenet
technology was used to recreate the
voice of tim shaw
an american footballer who is diagnosed
in 2013 with als a progressive
neurological disease that causes speech
impairment
tim was a particularly good candidate
for wavenet
a lot of people who get diagnosed with
als are asked to do some voice banking
where they'll record themselves so that
they can replay their voice in the
future like if there's a song that they
love to sing but not everyone does that
and with tim shaw he had a lot of
recordings of himself because he was
interviewed on tv
it's that amazing pre-game electricity
the butterflies are there and i'm ready
to hit somebody so you might want to
look out
researchers use 30 minutes of recordings
to create tim shaw's synthesized voice
unfortunately when tim sat down with his
family to hear his own voice for the
first time in years
he struggled to recognize it i know you
remember that
it had
[Music]
if you hear an old recording of yourself
when you were a kid
you're like did i sound like that but
other people do remember how your voice
sounded his family did i want to explain
to you why it's so difficult for me to
speak the diagnosis all of it it's his
voice that i've forgotten
his family cried because it's something
that's so personal it's such a key part
of your identity
there is still more work to be done to
make this technology more widely
accessible to als patients it's tricky
at the moment because the augmentative
communication devices that people like
professor stephen hawking used to speak
are generally not connected to the
internet and unfortunately these models
are far too large to be run locally on a
device so you must be able to quickly
send data through to a server to get
them to work that's a work stream that
definitely is being invested in and i
think in time these people will be able
to have these voices on their device so
that they can use to communicate every
day
[Music]
while speech synthesis is a very
personal way in which ai is beginning to
touch some people's lives there are
other projects concerned with something
that affects all of us
whether we like it or not
the weather
deepmind has recently teamed up with
researchers at the met office the uk's
national forecasting service perhaps
when people think of the uk met office
they think of michael fish this
meteorologist who famously predicted no
storm back before i was born this is dr
neil robinson from the met office neil
he is talking about an infamous case of
a weather forecaster who in 1987 assured
viewers that there was no hurricane on
the way good afternoon to earlier on
today apparently a woman rang the bbc
and said she heard that there was a
hurricane on the way well if you're
watching don't worry there isn't but
having said that actually
the great storm as it came to be known
turned out to be the worst storm to hit
southeast england in three centuries
nowadays of course weather forecasting
is based on phenomenally sophisticated
mathematical models that churn through
eye-watering amounts of data
we have one of the world's most powerful
supercomputers for analyzing the physics
of what's going on in the atmosphere to
make our weather forecasts
the halls where those supercomputers
exist their football pitch size
but these models do have their
limitations
so traditional weather forecasting
approaches have a real sweet spot about
a couple of hours in the future to maybe
a few days in the future
but a lot of decisions need to be made
on a shorter time scale than that
this shorter term weather forecasting is
known
as now casting
now casting is the problem of predicting
where is it going to rain and how much
just a short window into the future
this is the voice of deep ryan hadsell
so we're talking just is it going to
rain over my house in the next 30
minutes up to a couple of hours into the
future and predicting at a pretty high
resolution where is it going to rain
what are the real benefits of being able
to know what's going to happen in the
next hour the dream here is to be able
to warn people before really extreme
flooding events so that they can take
action like evacuation
there's been a few notable examples over
the last few years of these really
extreme rain events in the uk the
flooding at boss castle and coverage in
the south west it is these pictures now
with the vehicles bobbing around in them
just just floating along like corks
which
actually under climate change one of the
things that we're reasonably confident
is going to happen more in the future is
the rainfall is going to become more
extreme
the problem here is that the traditional
physics-based forecasting models involve
so much number crunching inside that
football stadium-sized supercomputer
that by the time their forecast is ready
it's already out of date
so researchers use other statistical
methods for their short-term forecasts
including a technique called optical
flow a computer vision method developed
in the 1940s which tracks the movement
of air over a two-dimensional image
it looks at the current state of clouds
and precipitation and then it tries to
follow those streamlines to kind of
extrapolate where it thinks those clouds
are going to go in the future it's not
an unreasonable place to start but it's
quite a sort of first order
approximation of the problem
and then one day raya hadzel was at a
chance meeting in exeter chatting to
some people from the met office
when she realized that this description
of clouds moving in a particular
direction across a screen
rang a bell
it was startlingly similar to a well
trodden problem in deep learning
video prediction is an area of research
where you take a video and then you just
try to predict what the next few frames
in that video are going to be
so if i see somebody
swinging a cricket bat
and then you stop that for a moment i
can sort of say ah what's going to
happen next is that that cricket bat is
going to continue to swing through
and you can think about rainfall as
being a video that's playing over time
where the radar provides this
information layer over a map of say the
uk as the rain moves along maybe a storm
comes up or a storm dissipates and so we
thought that doing that short-term
prediction into the future could be
solved by using video prediction neural
networks
but before the neural network could be
used to predict precipitation it needed
to be trained
for that the met office had their
rainfall radar a set of instruments
which use electromagnetic pulses to
measure the location and intensity of
rainfall
we got about a year's worth of radar
data across the uk
and turned this into something that
looked like a movie like a video playing
and we started training different types
of architectures to just predict the
next few frames of video
and this worked all right but what
tended to happen is that the neural
network just predicted a blurred out
future so we started looking at other
methods to solve this
and the method that has worked extremely
well
is to use a generative adversarial
network
this is usually talked about more in the
context of deep fakes because this is a
method that can be used to produce
extremely realistic fake videos
and this has been a really worrisome
actually use of ai technology
and so it was actually really nice to
see that this was an application of gans
a gan or generative adversarial network
is a clever way of having two neural
networks compete with each other to
produce the most realistic images
it's as though you have a pairing of a
counterfeiter and a police officer
the counterfeiter tries to produce an
image that will fool the police officer
and if it's not good enough they'll get
caught and have to try again
over time that competition gradually
increases the accuracy of those images
in this case those images are
predictions of weather in the near
future
and using this technique the results
were startling
instead of producing blurred out fields
of rain it produced very crisp lines of
rain and realistic movements of storms
across the uk
to test out exactly how good this ai now
casting was compared to the optical flow
method
researchers fed in a radar image of
precipitation patterns over scotland and
asked the neural network to generate
predictions of what the rain pattern
would look like
over the next 90 minutes
they compared these predictions to
observations of how the actual weather
turned out
when i first saw these images it was
unclear to me which ones were the
observations and which ones were the
predictions
i said
are these the same images they were so
close it was remarkable what you thought
someone had got mixed up and just given
you the same picture twice they looked
very similar
it wasn't perfect but it was very
realistic
the structure of these clouds ends up
being an important predictor of exactly
how heavy rainfall will be
where and when
and once that precipitation hits the
ground a different type of model takes
over
working out how water will run downhills
and collect in valleys potentially
causing flooding
one of the advantages of an outcasting
system like this is that it could mean
that the output is more useful for those
flooding models because
the actual predictions that make have
this more accurate fine structure which
means that when it goes into a flooding
model it hopefully could lead on to more
accurate flooding predictions
we're not necessarily quite there yet
with this system but it certainly has
moved us another step along
the gan model doesn't just provide one
prediction it can provide many different
estimates of what's going to happen in
the future
[Music]
and by inspecting those different
possibilities we can get an
understanding of what the different
extremes of the scenarios are which is
really valuable when we're trying to
help people make balanced decisions
about what they're going to do
the people who need to make the
decisions are met office meteorologists
they are the ones who assess all of the
information available and construct the
final forecast
neil surveyed them to find out whether
they preferred using the ai tool to
traditional methods
they really regularly chose this new
deep learning methodology over the
traditional methodology which is a
really good sign
the now casting project represents a
first step in how ai could be used in
weather forecasting
but there are still important challenges
to iron out for instance because these
machine learning models are based on
what has gone before
they're not good at forecasting really
unusual extreme weather events
as forecasters the more rare an event is
the more interested we are in
forecasting it and that's one of the
great things about the traditional way
we do weather forecasting i think it's
also why in the view of meteorologist
deep learning is never going to replace
the physics-based models
i actually think the future is really
for a hybrid approach where we're able
to take the physical knowledge and
combine that with the
power of deep learning methodologies
because of these limitations and the
fact that neural networks cannot explain
all of their predictions in detail they
aren't yet being incorporated into the
met office's official forecasts but the
collaboration with deepmind has provided
a glimpse of a future in which
artificial intelligence technologies
augment the capabilities of trained
meteorologists
[Music]
of course being able to make predictions
is useful for all kinds of real world
problems
but how does ai fare in a game that has
adored around the world
for its glorious unpredictability
last year deepmind published a paper on
how ai could transform football in
collaboration with liverpool football
club here in the uk
why liverpool you ask
well who's your favorite football team
liverpool i love liverpool i watch every
one of their matches
turns out that deepmind ceo derma
cesarebes is a lifelong fan of the reds
now who would have thought it
i know no one will believe this but they
approached us of course we jumped at the
chance and they happen to have one of
the best analytics teams in the world of
sport currently and of course we got a
tour of the training ground which we
needed obviously to have as part of the
collaboration were you free that day i
was happen to be free that day
miraculously
of course crunching data to analyze a
game like football is nothing new
what has changed in recent years though
is the sheer amount of data available
everything from computer vision
algorithms monitoring players positions
and motion sensors picking up on players
movements
carl turles one of the authors on the
football paper is based at deepmind's
paris office
over the next five years one of the big
ambitions of this football work is to
build a prototype of an ai system known
as an automated video assistant coach or
avac for short
[Music]
this is basically a system that
seamlessly integrates several data
modalities like raw video footage
tracking data event stream data all
sorts of sensors that the players are
wearing to assist coaches with their
decision making
there are a few different techniques
that are useful here there's computer
vision which can detect what's going on
in footage from a football game
then there's game theory which is all
about maximizing your advantage over an
opponent
and then there are statistical learning
methods which can hunt for patterns in
previous games
put them together and this automated
coach could make counter factual
predictions of what would happen in the
game if a particular tactical change is
made or a certain player is replaced
say for argument's sake that liverpool
fc are up against arch rivals manchester
city in a big premier league game
liverpool's coach could use the ai
system to monitor the match and provide
tactical feedback in real time
so a coach might say to the system hey
avac what will happen if i
move sella from
winning a position to a striker position
or we would move fabinho from defense to
midfield
so sort of this counterfactual
questions that are really interesting
for a football coach can we play that
out based on what we've seen in the
first half
the coach could then be shown a
simplified simulation a video with dots
moving across the pitch to indicate
possible player trajectories in
different scenarios
the idea here is not to replace human
analysts but to complement them with
another powerful analytical tool
the avac
is just going to give advice
and is going to say like what it
believes is maybe a good action to take
and in the end it's up to the coach and
it's of course also still up to the
players to act upon that
[Music]
it's not just during a game that such a
system could be useful it could help in
post-match training too highlighting the
exact moment when it would have been
better for a player to pass rather than
taking a shot at goal
although deepmind's research is
currently focused on new analytics tools
for coaches and teams
karl tells believes there are also ways
in which ai could enhance the experience
of football fans
currently when a fan watches a game on
tv there will be like
expert commentary but with new
technology this could become
personalized expert depending on your
own interests maybe what sort of
questions you would ask the ai about
your game for example on tactics
maybe in a more distant future fans will
have access to a screen in the stadium
or vr that augments their experience so
for example getting a feel for the pitch
[Music]
it's easy to see how a more personalized
experience for fans and improved
predictions for teams could have an
impact on football in future
but as i said before football is a
joyously unpredictable game
i don't think we will be able to predict
outcomes of a game accurately at any
point in time and this simply because
the decision making of pitch by coaches
and on pitch by the players is still in
hands of humans right so the signal is
still noisy
and there are problems with relying too
much on ai for what is at heart a deeply
human game
in 2020 the scottish football team
inverness caledonian thistle fc
announced that it would live stream its
games via cameras which automatically
track the football to give viewers the
best view of the action
during one game the automatic camera
seemed much more interested in following
the linesman around the pitch
turned out it had mistaken the
linesman's bald head for a football
and there are concerns that computer
vision systems like these might be much
better at tracking some players than
others
the current systems don't capture the
events that happen in women's sports as
well as they do for men's sports
here's jackson brochier another author
on the football paper so even where
we're trying to do proactive research on
women's data in an equal way to men's
data the labels that identify what's
happening in the videos that we use for
the training are actually much less
accurate
to those clued up about the problem of
bias in ai this might sound like a
familiar story in order to get really
good at analyzing the performance of
football players and teams an ai system
would need to watch hundreds of hours of
video footage from football matches
all of this data then needs to be
manually annotated to tell the system
what is going on in each frame
the trouble is when it comes to women's
football there is not nearly as much
annotated data to train on you might be
wondering why would the gender of a
football player even be relevant here
but as we've seen in numerous other ai
systems sometimes even small differences
perhaps the body frame of the players
could be enough to mean that the ai's
predictions on those games would end up
being less accurate
there is a broader point here currently
ai systems are only as good as the data
they're trained on which means if a
particular group is missing from your
data set the implications can be huge
when they first released phones that
would unlock from your face the images
they used to train those algorithms
if they were more people of white skin
versus black skin then it learned how to
identify those faces better
what we want to do on sports side is
make sure that the solutions that we're
developing are not biased to
gender or
skin tone or any other variances in the
volume of data that we're learning from
researchers are currently considering
technical solutions to address the lack
of data from women's football
but these are specific fixes to a much
larger problem of bias in ai systems and
as much as those working here believe
firmly in the benefits of deploying ai
in the real world there are also
potentially unwelcome consequences
to new technologies that have to be
carefully navigated
in the next episode of the deepmind
podcast we'll be taking a closer look at
the efforts at deepmind to make sure
that when ai reaches the real world it
works for everyone
we know that periods of history have
caused harms to specific communities
right and if we look at modern
technology through that lens we see very
similar patterns and certain uses of ai
and that is all for this episode gotta
run because the forecast says reigns on
the way i'll leave the credits to my
wavenet voice shall i
deepmind the podcast is presented by
hannah fry
special thanks for this episode go to
norman casa grande the engineer at
deepmind who found the time to create
hannah's wavenet voice
the series producer is dan hardoon of
whistle down productions
until next time goodbye
you


GPT 4 is Smarter than You Think: Introducing SmartGPT

I have three goals for this video first
I want to show you a way of using gpt4
to get smarter results second I want to
argue that The Benchmark results we have
for GT4 do not reflect its full
abilities and third I want to show you a
system that I am developing somewhat
cheekily called smart GPT that is
already showing significant results on
official benchmarks it remains to be
fully optimized which I think is
exciting in itself I have shown the
system to people at openai who have been
quite impressed and I'm going to end
with some Reflections on where that
might leave us for gpt5 but before I get
into how it works I just want to show
you one example of it in action to wet
your appetite this example comes from a
TED Talk that was released this week so
suppose I left to five close to dry out
in the sun and it took them five hours
to dry completely how long would it take
to dry 30 clothes GPT 4 the newest
greatest AI system says 30 hours not
good on the left you can see gpt4's
original answer and it gives this answer
pretty consistently whenever you prompt
it with the question provided on the
right you can see the final answer from
the smart GPT model which is correct and
it consistently gives that answer I
really like how it gives context as well
and it provides some of the assumptions
that it had in giving this correct
answer now don't you worry there will be
plenty more examples to go through in
this video including another one from
that Ted talk but first I want to give
you an overview of what is this smart
GPT model where did I get my inspiration
for it from and how does it work I'm
going to keep it fairly simple because
it's the beginning of the video and I
know a lot of people won't really care
about the inner details that will come
later in the video but the high level
overview is this there are at least
three things that have been proven to
improve the outputs of gpc4 what's
called Chain of Thought prompting
sometimes called step-by-step prompting
reflection or finding its own errors and
I did an entire video on this called
Gypsy 4 can self-improve and dialoguing
with itself entering into a back and
forth on its own outputs and deciding
which one is best you can see the title
of the papers which contain much more
detailed results of course linked above
now the first paper only came out a few
days ago Midway through my testing so my
results don't even reflect the full
capacity of the model and even if
there's nothing else you take from this
video the results from this paper can
instantly improve the outputs you get
from gpt4 many of you might remember
that prompting GT4 with let's think step
by step improves its results to give you
a very quick reference point just asking
a question to Gypsy 4 gives you 81
accuracy with that prompt let's think
step by step it goes up to 86 but
algorithmically the paper found an
improved prompt that can give you even
better results 89 accuracy all we do and
this is the first part of smart GPT is
we add answer let's work this out in a
step-by-step way to be sure we have the
right answer now I have so much to say
about why I think this works but I know
many of you won't be that interested in
my theories so I'm going to save them to
the end for those who are interested
some of you just want the results so I'm
going to get to those first so far you
might be thinking well thanks Philip
that's a cool prompt I'm going to use
that but what's this whole smart GPT
about is it just a single prompt no I
believe with evidence there are ways of
leveraging even better results than just
using a great Chain of Thought prompt so
let's move on to the next part of the
system these different outputs in the
middle for my tests I typically did
three outputs but of course depending on
the context window it could be far more
than that and I'm going to talk about
ways I could further improve this model
or we could later on in the video just
to restate these outputs are when you
take the user input and add the word
question at the start and then at the
end add answer let's work this out in a
step-by-step way to make sure we have
the right answer at this moment many of
you are thinking what is the point of
multiple outputs it's gpt4 it's just
going to give you the answer it thinks
is best and that's it well actually it
doesn't quite work like that these
models have a temperature between zero
and one I believe the default 4gbt4
might be around 0.5 and simplifying
massively this determines how creative
or conservative the model is in giving
its outputs so given that gpt4 tries to
be fairly creative you don't get the
same output every time the output is
randomly sampled according to an
internal probability distribution so you
can get situations and I face this
hundreds of times where some of the
outputs are correct and others are
incorrect and this is where reflection
comes in sometimes definitely not always
but sometimes quite often gbt4 can
detect the errors in its own output and
many of you will notice at this point
that the prompt that I used to elicit
gpt4 to spot its own errors contains the
same step-by-step prompt I used earlier
that has been shown to produce good
results so to summarize sometimes at
this stage gpt4 detects the errors that
some of its outputs have made definitely
not always there are certain questions
it just simply can't spot the error but
sometimes it can and then I get it to
engage in a dialogue using a format
similar to one in this paper published
last month it's a short dialogue and
this is the step I believe that can be
most optimized in the future I Envision
an entire Council of advisors made up of
gpc4 imitating mathematicians judges Etc
at the moment it's just being a resolver
and printing a final improved output
anyway I'm going to get back to the
theory later in the video because I know
some of you will be getting bored at
this stage and want to see more
practical examples and the results from
my Benchmark tests as I don't have the
gpt4 API key yes I had to manually input
each of these steps hundreds of times
waiting sometimes three hours between
each go because you can only do 25
messages every three hours on the left
you can see the three outputs when you
ask it to think step by step and then
you have the researcher step in the
middle and at the top right and finally
the resolver step notice here I was
using the original let's think step by
step because the paper hadn't yet been
published on improving that prompt but
it's time for the second example from
that Ted Talk and then I definitely will
get onto the benchmarks a different one
I have 12 liter Jog and 60 liter joke
and I want to measure six letters how do
I do it just use the six liter job right
gpt4 spits out some very elaborate
nonsense
foreign
of course I tested smart GPT with that
question and you can see the difference
between the original Gypsy 4 which gives
this incredibly convoluted bad answer
and smart GPT The Final Answer output
now at this point I know many of you
will be impressed but you'll be thinking
I don't have time to input things five
times well I'm developing a model where
it can all be done automatically here is
a preview of how it works but of course
at the moment it has to use GPT 3.5
turbo because I don't have the API key
of gpd4 the epic thing is this you just
ask a single question I've written ask
smartgpt a question and of course it
does take a little bit longer to respond
because it's doing five or six calls via
API but it does output the final answer
from the resolver step I will be honest
and say that GPT 3.5 isn't as good at
reflecting or resolving but this is an
example of a question where the original
chat gbt consistently gets it wrong and
smart GPT 3.5 gets it right using this
program remember all you have to do as a
user is type in a question as normal and
it goes through this entire five or six
step process behind the scenes by the
way this was a question from mmlu which
is a famous Benchmark which I'll get to
in a second here's one last practical
example before I get to that Benchmark I
know many teachers use chat GPT and GT4
to create quizzes for their classes and
here is the same question put through
Gypsy 4 and smart gbt the question is
create a high school algebra quiz with
five questions and answers and
explanations at the end now points for
spotting the difference but if the
teacher had handed out the original quiz
look at the answers for question five it
says the answers are 1 and 1.5 but then
in the explanation it gives the final
answers which are correct by the way of
three and 0.5 so that would really
confuse some students at the reflection
stage smartgbts spotted that error and
resolved it and as you can see the
answer for question five has the correct
answers straight away if at any point
you're wondering if I completed the open
AI chat topt prompt engineering course
the answer is yes but it didn't inform
too much of my thinking it was more for
beginners and I had already factored in
things like giving the model time to
think and writing clear instructions The
Benchmark that I chose to test smartgbt
on was the famous mmlu massive multitask
language understanding Benchmark as you
can see the state-of-the-art is indeed
gpt4 with 86.4 accuracy and you know
open AI think it's a big deal because
it's the Benchmark mentioned on the
front page of their technical report
without boring you too much I extracted
the questions from the test set of the
mmlu data file and I didn't pick the
topics at random I went for those that I
thought gpt4 would find the hardest
delving into the original mmlu paper you
can see that Gypsy 3 found formal Logic
the hardest scoring just over 25 which
is random chance it's a four question
multiple choice test so around 25 or 30
is pretty bad and notice they helped out
Gypsy 3 here they did it few shot
meaning they gave it five successful
examples before asking it a new question
it's the same thing they did with Gypsy
4 they did it five shot but just before
I show you the results there are three
things I want to mention here first I
was curious how smart gbt would do
without any help zero shot second I
wanted to do it zero shot because people
using gpd4 don't typically give five
successful examples before asking Gypsy
for a question they just want code or a
quiz or a poem or an example they don't
often provide five brilliant examples of
code before asking their question and
third if I can prove it works zero shot
then of course future refinements can be
made to push the results even further
and here are the results from the first
25 questions from the formal logic test
set of the mmlu I did many more tests
after this but you can see from this set
if you just to ask the question you get
a lower overall accuracy but of course
68 for GT4 is still a huge improvement
over gpd3s around 25 what happens when
you add let's think step by step which
as we know now isn't the fully optimized
Chain of Thought prompt well on average
you get around 74 75 that was 75
examples inputted manually and I still
have all the tabs open I'm keeping them
open because I'm compiling a spreadsheet
with the actual outputs but what did the
resolver get drawing upon gt4's ability
to reflect and engage in dialogue with
itself it got 84 now notice something
about that number gpc4 zero short got 32
of the questions wrong that was half to
16 after putting it through the smart
GPT system there was one question where
the resolver model gave both a correct
and incorrect answer but I'm counting
that as an incorrect answer for the
purposes of this test anyway from 30 22
to 16 incorrect that is a pattern that
stayed consistent throughout all my
testing that approximately half of the
errors that gpt4 makes can be rectified
if you give it the optimized
step-by-step prompt get it to reflect on
its results and get it to engage in
dialogue and decide on a final answer at
this point for those people losing track
of all the details I want to put into
context what resolving half of the
errors on mmlu might mean in the context
of the big picture here's Leonard Heim
an AI governance researcher suggesting a
score of 95 on the mmlu would be
reflective of agi-like abilities I do
think I have like a 50 chance like
within the next 20 years or so there
might be something will be my call in
AGI or a transformative AI what do I
mean by this well maybe we can measure
it on benchmarks there's like this
famous mmau benchmarks like yeah there's
something which like scores like 95 on
this going back to the results if a
smart gpt-like system can automatically
resolve half of the errors that gpt4
makes on the mmlu that would increase
its score from around 86.4 to around 93
which is not far off 95 remember his
prediction was a 50 chance in 20 years
I'm talking about g54 now for those who
are still skeptical I'm going to show
you plenty more results now and then
walk through the papers that give the
theory as to why this works one thing
that I forgot to mention earlier is that
the human expert level on the mmlu is
89.8 and that's taking the 95th
percentile of human test takers and
remember those are domain experts in
each of these subtopics what we're doing
is testing Gypsy 4 or smart GPT on all
of the topics simultaneously so even if
smart gbt like systems can't quite reach
95 and I think honestly they'll get
pretty close with all the requirements
I'm going to suggest I think they should
almost certainly be 89.8 which is the
human expert test taker level intrigued
by these results I then put it through
the college math test from the mmlu and
remember this was before using the
optimized version of the step-by-step
prompt obviously I'm not going to go
through all the questions here but let's
skip to the final results we have zero
shot accuracy 6 out of 15 which is 40
the average when you add let's think
step by step was 53.5 and then the final
output of the resolver model had a 60
accuracy so it couldn't quite resolve
half of the errors but the overall
pattern held up in case anyone is
wondering about methodology I kept the
formatting identical for every question
I always opened a new tab for each
question it wasn't looking at the
context of what it had already put out
each attempt was fresh aside from the
resolver model which looked at the
context of the researchers output and
again as you can see from example 14 it
wasn't like the researcher could always
spot the errors or that the resolver
could always pick the right option
sometimes the let's think step-by-step
prompt gave the right output but the
resolver couldn't quite distinguish it
the optimized prompt gets a slightly
better output and upon reflection the
researcher can sometimes but not always
spot the errors of those outputs and
sometimes but not always the resolver
can swap based on those flaws which
answer is best these are incremental
improvements sometimes GT4 simply can't
get it right I have noticed a few themes
in those questions anytime it comes to
division multiplication characters or
counting in general deept4 tends to make
mistakes that neither the researcher nor
resolver can spot of course integrating
a few tools via API would likely solve
those issues and I don't want to preempt
the conclusion to too much but I believe
a smart gpt-like system with tools
integrated could probably score around
95 right now on the mmlu especially if
it was helped out with a few shot
prompting to add weight to that
preliminary conclusion I tested it on
certain topics and had to stop because
it simply got the questions right every
single time for example High School
psychology on the mmlu I then tried
pre-history which it also aced before
finding machine learning where I got
more interesting results zooming in this
time the raw score was 65 The Chain of
Thought let's think step-by-step average
was 71.6 and the resolver model got 80
let's now look a little deeper into why
all of these steps might improve the end
result in reply to the original let's
think step-by-step paper which was
published around a year ago Andrei
carpathy said this adding something like
let's think step by step to the prompt
is a way of using the input space for
computation that you'd normally want in
the hidden state of the model instead of
the workings out being done in the
activations of the neural network it's
done in the discrete tokens of that
input space and he adds did not super
see this coming and here is the paper
released three days ago that improves
upon that original prompt they also did
their testing zero shot like me and they
tested many prompts starting like I did
with just direct prompting just asking
the question like 99 of users would do
of gypsy 4. and then they tried like me
the well-established let's think
step-by-step prompt they also
iteratively tested Seven original
prompts as well as the prompt that I've
now integrated into smart GPT the let's
work this out in a step-by-step way Etc
they share my opinion that zero shot
prompting setups have the benefit of not
requiring such task dependent selection
of exemplars you don't have to find
correct examples it just does it all for
you here are the end results for gpd4
that we saw earlier showing the
difference between asking directly your
question and using these refined prompts
notice that this technique is somewhat
model dependent and it doesn't have the
same effect on smaller or weaker models
before we move on to the next paper
there is one somewhat failed prompt that
I want to pick up on it's this
self-critique prompt where they ask
answer the question then critique the
answer based on the critique we consider
the other answer options and give a
single final answer and you might wonder
why didn't that prompt perform best when
we know that reflection and dialogue can
work my theory is because it's trying to
do all of it in one prompt through my
hundreds of experiments I've noticed
that gpt4 can only handle so much in one
go it simply gets overwhelmed or
confused if you ask it to do too much in
one prompt that's why I broke my model
into stages to allow it to show off each
of its abilities one by one and before
we get to the other papers what's my
personal Theory as to why this
eliminates up to half of the errors that
gpt4 makes well my guess is this
remember that gbt4 is drawing on a vast
data set of internet text and let me ask
you what kind of text has things like
question answer let's work this out be
sure we have the right answer the kind
of data that would have that text would
be things like tutorials or expert
breakdowns so I believe you're
triggering more of the weights inside
gpt4 that relate to things like expert
tutorials and so inevitably you're
getting slightly better answers next
I've already explained why you'd get
different outputs when you give the
exact same prompt that's down to
sampling and the temperature of the
model but to simplify massively
sometimes Gypsy 4 will give you an
output that it knows isn't the most
probable it introduces some Randomness
into its sampling by generating multiple
outputs you're getting a larger sample
size reflecting the full range of
probabilities that gpd4 subscribes to
its outputs you're reducing a little bit
some of the randomness that's inherent
in gpd4 outputs next I believe that gpc4
can sometimes spot its own errors
through reflection because prompting
like this triggers a different set of
Weights you could almost think of it as
a different mindset one more focused on
finding errors again if the question is
too hard or involves counting characters
division multiplication as I said
earlier this won't help but a percentage
of the time it can spot its own errors
and point them out notice this is a
separate bit of inference not lumped
into the original prompt and when it
does successfully point out the errors
it can often engage in this dialogue
with itself notice in a meta kind of way
I'm using the step-by-step prompting to
improve the reflection and dialogue so
those are my theories as to why it works
but at the end of the video I'm going to
show you at least five ways I think the
model can be further refined before we
do though I looked up the paper by Zhou
which produced that prompt that did the
best in the previous paper they came to
that special prompt through automatic
prompt engineering but there's something
interesting I want to point out though
on page seven they say we use automatic
prompt engineering to find a prompt
starting with let's that maximizes the
likelihood of correct reasoning steps
then they found the best one that I
integrated into smart GPT let's work
this out in a step-by-step way to be
sure we have the right answer that's the
one I want you to use and they ran their
own benchmarks and of course it did
improve the scores but the interesting
thing to me is they started with let's
each time so even that first stage for
the model might not yet be fully
optimized maybe there's a prompt that
doesn't begin with let's that improves
this initial result still further anyway
back to the papers I know many people
watching this will wonder if I read the
paper boosting theory of Mind
performance in large language models via
prompting and yes I did because they
tested something similar for a theory of
Mind test using similar techniques they
were able to get theory of Mind accuracy
for GPS 4 from 80 to 100 and they
conclude that these results demonstrate
that appropriate prompting enhances
large language model theory of Mind
reasoning and they underscore the
context-dependent nature of these models
cognitive capacities they use that
original prompt let's think step by step
along with some few short examples take
a look at the gpt4 table and you can see
how the let's think step by step
improved the results dramatically and as
I theorized earlier adding few short
examples would push this still further
this is part of why I think that 95
barrier on the mmlu will be broken
probably this year by gpt4 a few other
points from this paper they admit that
there is not currently a theoretical
understanding of why these prompting
techniques are beneficial I've given you
my theory and carpathies but no one
quite knows for sure lastly from this
paper and I found this really
interesting giving it generic few shot
prompts that weren't directly there
theory of Mind actually improve the
outputs slightly more than giving it
direct theory of Mind examples this
opens the door to the first of the five
ways I anticipate smart GPT getting even
smarter it could be possible to come up
with generic few shot prompts that could
be automatically integrated into the
model that don't necessarily relate to
the topic at hand this graph shows the
impact of adding few short examples to
gc3 and if this can be done in a generic
way for gpd4 results could be improved
still further next the boosting theory
of mine paper speculates that
integrating some of these approaches
could boost the performance of weaker
models to beyond their levels of GPT 4's
zero shot accuracy next here is the
original Dira paper that inspired me to
have the researcher and resolver
dialogue at the end of smart GPT as they
say the dearer approach shows
significant improvement over base gpc4
performance and these were open-ended
questions by the way not multiple choice
so this is more generally applicable
than you might think you can see from
this table how results improved after
engaging in this dialogue and that
brings me to the second way I anticipate
smart GPT getting smarter in the future
a longer and more Rich dialogue at the
moment we have this simple research and
resolver two-step dialogue I can imagine
a council of advisors you can imagine a
mathematician chipping in and a
philosopher and a professor each one
tapping into slightly different weights
of gpd4 extracting more hidden expertise
I'm not saying that would transform the
results but it might Edge them another
few percent higher next even with longer
dialogues and different experts we could
find ways of optimizing these prompts
just like we did with the original let's
think step by step that's the Third
Avenue of improvement that I envisage
because I came up with these prompts I'm
sure they could be improved next we
could experiment with different
temperatures remember a lower
temperature makes the model more
conservative relative a Higher One
towards one makes it more creative we
could experiment with a higher
temperature to produce a more diverse
range of outputs at this stage and then
perhaps a more conservative
deterministic temperature for the final
judge or resolver it might not work but
it's worth trying and the fifth
Improvement I know would work
integrating apis for character counting
calculators code interpreters Etc
spending these weeks manually sorting
through the outputs of GT4 on these
benchmarks I can really see where it
goes wrong and it's often by getting
letters in the wrong order or making
mistakes with division it gets the high
level logic right and then makes quite
simple errors basic tool integration
would I am sure push the results still
higher now I know this isn't my usual
video and trust me I have been following
the AI news and we'll get back to that
very soon I'm determined to make those
improvements and push smart gbt even
further but of course that would be
aided massively by getting access to to
the plugins and the gpt4 API key so far
I've had to do all of this manually
which was a lot of work now as you saw
earlier I have drawn on gpt4 to help me
develop a program in replit to automate
this process but at the moment it's GPT
3.5 and honestly the context window
really limits the ability but I do look
forward to the day when I can integrate
gpt4 and put this out as an automatic
model for people to test and play about
with I'm sure that something similar
will ultimately be incorporated by
openai itself maybe as a thoughtful mode
or smart mode a bit like Bing has
creative precise balance Etc each
response does take longer but as you've
seen the outputs are noticeably better
if the results of models like this one
do officially exceed the 86.4 that
openai talked about in the gpt4
technical War I do think that would
reveal quite a few things first the
openai isn't even aware of the full
capabilities of its own model I don't
even know if they anticipated things
like Auto GPT I do think it would reveal
that they need to do far more proper
testing of their models before they
release them they should make
falsifiable predictions about what their
models won't be capable of that way we
would know just how much they know about
their own models what we're trying to
avoid is a situation where open AI say
their model can only achieve X and then
when they release the model in the wild
someone comes along and achieves why
where Y is much more impactful than x so
those were the goals of this video to
show you how to get more out of GT4 to
run you through some of the fascinating
papers that have been released in the
last few days and weeks the third goal
was to show you what this model could do
with some official benchmarks and
suggest ways it might get better in the
near-term future of course if you have a
gc4 API key or are an expert in
benchmarking systems like gpc4 I'd love
to hear from you I guess the final goal
was to perhaps suggest to you that
openai don't know as much about their
own models as they might lead you to
believe thank you so much for watching
to the end and have a wonderful day


Fair for all - DeepMind: The Podcast (S2, Ep8)

[Music]
this is deepmind the podcast
i'm hannah fry a mathematics professor
and your trusty guide to the rapidly
evolving field of artificial
intelligence
so far in this series i've spoken to
people who are using ai to advance
scientific discovery you have techniques
that can tackle some of the hardest
problems that have eluded the brightest
minds attempting to solve intelligence
do people still roll their eyes every
year it becomes lease
and using ai for real-world problems
like weather forecasting the dream here
is to warn people before really extreme
flooding events
each of these tasks is enormous in its
own right
but there's a fundamental challenge
running in tandem
ai needs to benefit everyone not just
those who build it
[Music]
deepmind's mission statement is solving
intelligence to advance science and
benefit humanity
we really need to take
that mission seriously and to ask
questions about what does it mean to say
advanced science and humanity
for shakir mohammed a senior research
scientist the only way to live up to
that is to create ai that works for all
of society
the last set of people who said they
were going to advance science and
humanity created colonialism they
created a structure of division in the
world we cannot be complacent about what
it means to put forward a mission like
that into the world
this is episode 8 fair for all
let's start at the beginning if the goal
is to build ai that's fair for everyone
what does it mean to be fair
you might imagine this is a question
with a simple answer fairness surely is
just about ensuring that one group of
people doesn't end up disadvantaged over
another
unfortunately once you try to put that
into practice it quickly becomes clear
that sometimes competing definitions of
fairness can contradict one another
let's take gender as an example
until a few years ago if you searched
for the word ceo on google images you'd
just get images of all white men ceos
this is sasha brown from deepmind's
ethics team she probably like most
people thinks that an image search
showing no credible female ceos at all
is not exactly an example of a nice fair
algorithm
people then think that a ceo has to be
white and male but also because of the
way that the algorithm works whatever
comes up on the top page is more likely
to be clicked on and therefore is more
likely to stay near the top page so
you're kind of reinforcing this
stereotype
now there are a number of ways that a
researcher could alter this search
algorithm to make it fairer
for example they could tweak the first
page of results to achieve a more even
gender balance 50 female 50 male say
over 50 of people in the uk are female
but seven percent of ceos in the uk are
female so should you have seven percent
it's not a simple question as to like
how fair is fair enough so if i google
ceo now i'm looking up now
oh okay things have changed then because
let's look i would say it's
predominantly white male but on the
front row the third image haha there is
a white woman and a black man actually
there's pretty good representation
across the board there is i don't know
about your research but on mine there's
hot ceo romantic novels
yeah i've got that
steamy
in this example then some slightly dodgy
romantic fiction aside the search
algorithm is not designed to accurately
represent company leaders as they exist
but to convey how they might look if
company leaders were more representative
of society as a whole
this is an active choice
sometimes you don't want your algorithms
to be a mirror you want them to reflect
a version of the world you'd like to
live in rather than the one that you do
because a very real danger of unfair
algorithms is that they can serve to
perpetuate stereotypes or biases about
certain groups in society locking us
into a cycle that's difficult to break
if you cast your mind back to earlier in
the series you'll remember lisa ann
hendricks a researcher on the language
team while studying for her phd lisa ann
was co-lead author on a rather
brilliantly titled paper women also
snowboard the study looked at algorithms
which automatically generate a caption
for a given image this is a car this is
a dog that kind of thing
lisa and her colleagues realized that
when these image classifiers were shown
images of people snowboarding
they were much more likely to label the
person as male
even if the person in the image was in
fact female or their gender was unclear
i'm just sort of imagining you in grad
school right you'll sit in front of your
computer with loads of pictures of
female snowboarders and it's just
labeling them i mean was it every single
time man snowboarding man on a snowboard
jumping
not always but frequently
the opposite was true for images of
people in kitchens who were more likely
to be identified as female
it turned out that the image classifier
was focusing not on the person in the
image but on their surroundings applying
a pattern it had picked up from the data
that men were more likely to appear in
images with snowboards
when you trained models on this data it
magnified and amplified this bias and so
whereas at train time maybe you had a
skew of 70 30 men women snowboarding now
at test time you have 90 10.
what lisa ann is referring to is a
phenomenon known as bias amplification
somewhere in the algorithm for reasons
though hard to pin down
any initial unfairnesses get exacerbated
and those initial biases tend to arise
during what's known as train time
when a huge online library of images is
manually labeled and categorized by
humans
humans go on to a crowdsourcing website
and write sentences that describe the
images so you had some images where the
person was very obscured you know really
wasn't clear how anybody would make a
judgment on something like gender and
they would just say man because they
themselves had a stereotype of who
snowboards which is another place where
bias can arise which is that a lot of
times we think that humans are going to
give us ground truth and when you're
talking about anything sensitive it's
not always clear what the right way to
label something is
once they recognize the image
classifiers were biased lisa ann and her
colleagues shifted the algorithm to
focus on the subject of the photograph
rather than their surroundings
but the truth is there are no easy
answers here
de-biasing detoxifying or weeding out
stereotypes from ai systems
is much more than just a technical
challenge that can be solved with a
corrective algorithm
and you won't be surprised to hear that
it's not only gender bias that crops up
in ai systems but racial bias too
there are countless examples of image
classifiers which completely fail to
recognize people from particular racial
backgrounds
in 2020 there was an outcry over an ai
tool which takes pixelated images and
converts them into high resolution ones
it had been disproportionately trained
on images of white faces a bias it duly
demonstrated when it transformed a
pixelated image of former u.s president
barack obama
into a picture of a white man
another example that has come up is an
ai facial recognition technology
incorrectly identified 28 members of the
us congress
as people who had been arrested for
crime this is auburn aka global
education lead
the first matches we are
disproportionately
people of color imagine if such a tool
was deployed at scale it means that
black people like me will be walking on
the street and maybe suddenly stopped by
the police who have identified me
through a tool that tells him or her
that i'm a criminal
these kinds of biases are a real concern
particularly when they appear in
predictive algorithms where historical
data is analyzed and used to predict
what will happen in the future
a recent study found that an algorithm
used on millions of patients in america
to identify which of them were eligible
for an advanced healthcare program was
biased against african american patients
the algorithm was told how much money
had previously been spent on a patient's
healthcare expenses and use that to
predict their future needs
that idea might make sense on the
surface but the algorithm was found to
be disproportionately rejecting black
patients from the advanced healthcare
programme
the data didn't have to have any
information about race in order to have
racist outcomes
sasha brown from the ethics team again
so the algorithm based its decisions and
it assigns risk scores to patients on
the basis of the total health care costs
they'd accrued in the year before and so
they assumed that the higher the total
healthcare costs the higher the need of
the individual
and they even checked and they found
that the average black person in the
data set had the same average healthcare
cost as the white person in the data set
so they were like okay this is fair
but then on closer inspection they saw
that the average black person in the
data set had also been sicker than the
average white person in the data set and
so actually it was unfair and it was
discriminating against black people
historic disparities in access to
healthcare meant that less money had
been spent on black patients with
similar needs to white patients the
algorithm falsely concluded that black
patients must be healthier than white
patients and refuse them advanced care
it's an interesting example because
people tend to think okay well we've
removed the race bit of the data set and
anything that we think is a proxy for
that and so it's not going to cause harm
but actually if you do systematic
reviews of algorithms you see that in
the real world it is actually having
impact where historically
underprivileged groups are getting less
help as a result of this
[Music]
for some these examples of how ai can
discriminate against particular groups
are not aberrations but part of a
broader historical trend of how the
negative impacts of new technologies are
disproportionately felt by society's
most marginalized groups
we know that periods of history have
caused harms to specific communities
right and if we look at modern
technology through that lens we see very
similar patterns and certain uses of ai
william isaac is a senior research
scientist on deepmind's ethics team
if you listen to series 1 of this
podcast you might remember william's
work on the use of racially biased
algorithms in the u.s criminal justice
system
in may last year william was one of the
authors of an influential paper on d
colonial ai
can you give me an example from history
where science and technology has been
used to exploit marginalized communities
the most salient one for me is the u.s
public health studies that were done
from the 1930s all the way to the 1970s
on african-american sharecroppers
in 1932 scientists began a study of
syphilis in tuskegee alabama
at the time there was no known treatment
for the disease
600 african-american men were recruited
for the study in return for free trips
to a physician and hot food
of those 399 had latent syphilis but
none were told of their diagnosis or
what the study was for
only that they had bad blood
[Music]
over the following four decades the
scientists wanted to study how the
untreated disease would evolve
about halfway through the experiment
penicillin became the standard treatment
for syphilis
it was almost totally effective against
the disease but the scientists did not
offer it to their patients
even as many of the men went blind or
developed severe health problems the
scientists continued their experiment
by the time the story broke in 1972
28 of the men had died from syphilis
while around 100 others had passed away
from related complications
this is one of the black marks in
history in the scientific community
because it not only represented
the kind of power imbalance that you
know science can have in communities but
also it eroded trust in the
african-american community
for william a decolonial lens reveals
clear parallels between the treatment of
black communities by today's biased
healthcare algorithms
and the medical researchers of 1970s
america
being conscious of these historical
parallels is one thing
but d colonial ai urges those involved
in designing ai not to repeat the
mistakes of the previous technological
revolutions
the decolonization of ai is also a
project of hope
shakia mohammed was a co-author of the d
colonial ai paper
it is a project to say that we can come
from a world that was built on slavery
and genocide on exclusion and
discrimination but actually we can try
to design a better world that's my
number one hope that everyone they don't
have a reaction to this word
decolonizing but they actually see that
it is asking us to think of a different
future than the one we've inherited
for william and shakir building ai that
benefits all of society goes beyond
fairness in the narrow sense of
achieving equal algorithmic outcomes for
all groups
it involves thinking hard about where
why and in which circumstances ai
systems are deployed
take the example of a facial recognition
system for entry into an apartment
complex where the majority of residents
are black
because facial recognition systems are
traditionally less accurate for people
of color
residents suspected that developers were
trying to attract more white higher
income residents into the block
they successfully prevented the facial
recognition system from being installed
and so this the intentional acts of how
we use these systems is also equally as
important as whether or not the system
meets this goal of having equal
performance across groups so if you just
look at the system whether it's fair or
not you would miss the entire point of
where the negative externalities of a
system lie
how
long do we have to wait
until people stop making the exact same
mistake
over and over again
i hope not too long even if the first
step is like we don't release systems
out into the world that actually impact
people until we address the kind of
underlying challenge around data that
would be a good first step right i know
we're not going to solve all the
problems with embedded buys and data i
don't think that's an attainable goal
but we can control which ones are
released out into the world in high
stakes settings like whether or not
you're going to be able to stay with
your kids or whether or not you're going
to have housing or whether or not you're
going to go to jail
but as the example of the housing
project demonstrates the issues around
consent and data privacy are becoming
all the more important as ai is
increasingly deployed in the real world
and in some cases the question of what
an ai claims to know about you
and who it shares that information with
can have serious consequences
take for example a notorious paper from
stanford university in 2017
which announced an algorithm that could
accurately predict whether someone was
gay or straight from a photograph of
their face
the algorithm in question had been
trained on thousands of pictures of
faces from a dating website together
with their sexual preferences and
boasted 81 accuracy in identifying gay
men and 74 for gay women
there was just one problem
trying to determine someone's sexuality
from their facial features is what's
commonly known as junk science
it's hard to even know where to begin
with that paper this is kevin mckee a
senior research scientist we met him in
episode three where he told us about his
algorithm for altruism
last year kevin co-authored a paper
about the impact of ai on queer
communities
you could potentially be falling into
the trap of thinking that there's some
sort of causal link between someone's
appearance and the way that they
identify and again it's leading you down
a very dangerous path
what might seem like a silly gimmick to
some people
could have grave consequences if it were
deployed in some parts of the world
here's william isaac again
there are governments who would likely
use a system even one that was
completely inaccurate and just junk
because there's a belief that machine
learning systems confer a sense of
authority about information and
prediction and this is why these are
particularly pernicious systems
and although these so-called gaydars are
fake pseudoscience
there are other ways that an ai tool
could genuinely threaten the privacy of
lgbtq plus people
every time you like a page or you
interact with someone else online you
generate digital traces the challenge is
when you have an algorithm that can kind
of sweep over all of those different
traces and start to say things about me
that i never intended to be public so if
i did not want other people to know that
i'm queer but then you know someone
designs an algorithm that's able to
predict that that's a huge challenge to
privacy
the
harms against queer communities i think
take many different forms today shakira
mohammed is also a co-author on the
fairness for queer communities paper
an everyday experience is being in an
airport and you are in those scanners
one really significant harm to our trans
community is that those scanners require
the operators to click beforehand male
or female
what shakira is referring to here are
the full body scanners that use x-rays
to look underneath a person's clothes to
detect any illicit items
an airport security officer might assume
that someone is male click the
corresponding button at which point the
scanner alerts the security guard that
this person has something concealed in
their chest area
but say this person is undergoing a
gender transition
they've been singled out and humiliated
by a machine that expects a man's chest
to look a certain way
that is a form of dispossession and that
kind of benign automation is something
that really affects people on their
day-to-day living experience
and as much as ai can threaten the
privacy of queer people there are ways
in which a carefully designed ai can
have a positive impact too
for instance in march 2021 a charity
which supports young lgbtq plus people
in america known as the trevor project
unveiled a chatbot named riley
riley simulates a conversation from a
young genderqueer person in crisis
the idea is that conversing with riley
can help a volunteer understand how best
to respond
honing their skills for the real
conversations they'll have
when they matter the most
so far in this episode we've considered
how researchers think about ideas of
fairness and decolonial ai to ensure
that their systems don't discriminate
against particular groups
we've also looked at some of the issues
that can arise from the deployment of ai
systems in the real world
but an important part of ensuring that
the benefits of ai spread to everyone
is making sure that more diverse groups
of people are present in the labs where
ai is built
how you actually get to really safe and
robust technologies is you have
different lived experiences looking at
the same technology from different
lenses and are empowered to actually
make sure that you live up to the kind
of building of like having technology
that is safe especially when you're
working with transformative technologies
that will impact billions of people
and there are no prizes for guessing
which groups of people are currently
underrepresented in the field of ai
one way to think of it is
how many
people are graduating from ai phd
programs
that's album aka again in north america
for example
it's estimated that the female graduates
of ai phd programs have accounted for
less than 18 percent
of all phd graduates on average in the
last couple of years as a father of
three beautiful girls that gives me a
lot of concern
the other group i swear that is
predominantly underrepresented is black
people like me
and other minority ethnic groups here in
the uk in 2019 black people made up only
three percent of the tech workforce
listening to this you might be wondering
about a question the auburn gets asked a
lot about the ai industry as a whole
why can't we just diversify that's what
every organization should have done from
the word go
one of the biggest challenges is
recognizing how systemic these issues
are
so is it that there aren't enough people
from the underrepresented groups that
album talks about who are trained in ai
to a high level there's always an
argument in the field around oh the
pipeline doesn't exist so you can
actually find people to hire and all
that i don't think that's true let's get
as many people that are already out
there let's change our recruitment
processes and all these things we can do
to tap into the untapped talents that
already exist out there but also while
acknowledging that there is actually
more work that needs to be done to
invest in the longer term pipeline
people from underrepresented groups
often don't make it as far as pursuing
phds in ai
in fact they drop out much earlier
here in the uk a government study found
that female students are much less
likely than their male counterparts to
study science technology engineering or
maths at a level other studies have
found that black students are more
likely than other ethnic groups to drop
out of these subjects at all stages of
their educational career
and while you don't always need a phd in
ai to work in the field
a background in one or more of these
subjects is a useful asset for an ai
researcher
it's this principle of making people
aware of the possibilities of a career
in ai that guides deepmind's own
education program
we've recently launched a program called
the deepmind academic fellowship to
actually support
early careers such as
to pursue postdoctoral studies in the
field of ai and then below that at the
post graduate level we've created the
scholarship program
which provides a very general
scholarship to students to pursue
masters and phd degrees in universities
aubum ekeke has seen first hand how the
promises of technology can inspire
people's careers i grew up in a very
rural and poor village in nigeria and at
the time i was in secondary school there
was a boy about my age who went to the
city on holidays
and one of the days we were playing
and he said to me he saw
this car in the city
and somebody tried to touch the car and
the car started shouting all of a sudden
and he said to me that you could
actually go to university and study the
thing that made the cartoon shout
and then he said so many other things
this thing can also help you to cure the
disease
later i realized that what this boy was
talking about was just a remote control
right so which has nothing to do with
computer science but what that did for
me was it actually helped me to be very
interested in studying computer science
at the time to solve problems that
existed in my community
as an example of the sorts of benefits
that ai could bring aubum told me about
a mobile app developed in 2018
by penn state university in
collaboration with google
the app uses ai to help farmers diagnose
diseased cassava crops
cassava provides food for over half a
million africans daily my parents used
to be peasant farmers so i used to go to
farm a lot the challenge we had with
cassava
is that they are mostly affected by
viral diseases which makes them
eventually inedible
a group of researchers developed a
solution using machine learning
that could help farmers better identify
and manage the diseases that affect
these cassava leaves by just waving
their mobile app on the cassava leaf
that's exciting for someone like me
right seeing that this kind of solution
could be deployed across africa
technology capable of making a profound
difference to countless communities
already exists
but there is a conspicuous lack of a
major ai presence in many countries
around the world
this is a fact that hasn't been lost on
shakir muhammad
in 2016 while attending a prestigious ai
conference he and some fellow south
african ai researchers started asking
themselves a question
why are there no other south africans at
this conference why are there no
africans to begin with at this
conference and actually maybe part of
the question is we don't need to see
them at these kind of conferences there
may be other new spaces we could create
that doesn't need to necessarily always
comply with the mold
it was this germ of an idea that led to
shakir and his fellow researchers
creating the deep learning in dhaba a
non-profit organization whose stated
mission is to strengthen the machine
learning and ai community across the
continent of africa
in dhaba which is a zulu word for a
gathering or meeting has been running
for around five years and led to some
impressive projects
take for instance the masakane nlp
project the name means we build together
in zulu and began when african
researchers recognized that none of the
current language models those gigantic
tools powering smart speakers and
chatbots could recognize african
languages
[Music]
one of the challenges is that there
aren't
language data sets for many african
languages or 4 000 of them the diversity
of language means that there are very
small communities of speakers of those
languages
local linguists are collaborating with
ai researchers to gather oral and
written data sets in several african
languages including swahili
in kenya for example
if ai is going to enter this world of
finance what they call fintech it
matters that that fintech development
product for mobile payments is in
swahili we have serious problems today
across so many of our countries
particularly in southern africa related
to the hiv pandemic one solution as the
need for advice for hiv but other areas
of maternal health care for example have
come on board is to create these kind of
mobile chat healthcare advisory it
matters that that advice comes in the
languages that people speak because the
ability to understand the true context
matters it's a different form of
representation which is why the kind of
narrow idea of diversity of the
workforce is always incomplete
as we've heard throughout this episode
the challenges of bringing the benefits
of ai to all of society are significant
individual algorithms need to be fair
and ai needs to create local benefits
for everyone not just those who live in
countries with a strong research
presence
addressing this problem will require a
more diverse workforce a more democratic
spread of ai expertise across the world
and the inclusion of a broader set of
perspectives in building products and
algorithms
we have a great deal of power because we
are the ones who decide what the data
set is if we decided to create a
different kind of data set a different
kind of prediction gets made
we can humble ourselves as researchers
to leave the bubble that we are in and
go to different kinds of communities to
come together as new kinds of grassroots
organizations and they can create a
powerful pressure on the world around us
initiatives that deepmind researchers
are working on from indarber to the
scholarship program are a start
but no one company can tackle this alone
we need to partner with other
organizations we need to work with
universities the government has a role
to play we need to work with a
cross-section of stakeholders to
actually
collaboratively
make progress
ethics has to be the input into what
tech we build and not an afterthought
and it needs to be all researchers
responsibility to think about the social
impact of their tech
it's not going to happen by default it's
going to take loads and loads of hard
work i'm excited to see
what does technology look like when you
have everyone involved in the process
what new ideas do you develop because we
won't have the same kind of clustering
effect of the same four or five ideas or
uses for technology they will look
different right and that's a good thing
and i think ultimate is the future that
i believe will get with time
we'll be looking towards what the future
holds in the next and final episode when
i sit down for a rare interview with
deepmind's co-founder and ceo demis
osabis the outcome i've always dreamed
of is
agi has helped us solve a lot of the big
challenges facing society today
you've been listening to deepmind the
podcast i'm hannah fry and the serious
producer is dan hardoon at whistledown
productions we'll be back soon