Governing Transformative Artificial Intelligence by Markus Anderljung

great great seen you all today so I'm
here to talk about sort of some slightly
more big-picture issues then you will
have heard about so far and so I'll be
talking about basically the issue of how
do we make sure that artificial
intelligence in the coming few decades
benefits humanity as much as possible
that's sort of one of the issues that
sort of me and my colleagues at the
Center for the governance so they I at
Oxford University is is really really
focusing on so we've already seen and
we're currently seeing a lot of sort of
big interesting developments and
artificial intelligence ranging from
sort of alphago to sort of opening eyes
stack till two more sort of real-world
applications that europe may be seeing
around this conference of sort of
applying artificial intelligence to
real-world problems such as medicine etc
however the relief sort of big the
large-scale changes that we're going to
see and sort of the potential benefits
and the potential risks of artificial
intelligence likely lie in the future
I'll try to illustrate this with what I
think is sort of one of my one of my
favorite graphs or like one of the most
sort of the most important graphs out
there which looks like this this is a
graph that was produced by in a paper
that we produced with with some other
people in this general AI governance
space so you see on this graph is on the
y-axis you see the probability that
people assign to getting human level
machine intelligence so these this is
sort of in layman's term this is when
sort of artificial intelligence systems
will be sort of as smart as humans and
on the x-axis you have sort of years
from 2016 when we did this survey I like
when do you think that there's above 50%
chance or when do you think that you
will get human level machine
intelligence and the data here doesn't
come from from anybody it doesn't come
from from sort of people on the street
but rather it comes from sort of experts
in this field so we went to the sort of
top conferences in machine learning ICML
and nips and we basically asked them
this question there's a bunch of things
they're really interesting in this graph
I think a few things to point out is the
first one is if you can see the little
sort of very weak gray lines these lines
show sort of individual respondents
their responses and the really
interesting thing is that they're
incredibly disparate they're everywhere
so there's apparently there are some
people who believe that sort of within
the next about ten years there's a
hundred percent chance that we will have
reach a tell mi others they give
responses that look like there's
basically just hugging the x-axis they
think there's basically zero percent
chance that we with it within this
century within the coming hundred years
we will have reached human level machine
intelligence so absolute of the first
conclusion incredible sort of amounts of
uncertainty here or people have an
incredible amount of disagreement but
then the second thing is when you look
at the aggregate it looks like we get
sort of what to me seemed like pretty
high numbers if this number is true if
it's true that we have like a fifty
percent chance of getting to human level
machine intelligence in the next fifty
years that's a really big deal that's
something that's likely to change
society in all kinds of ways that's
going to be difficult for us to foresee
and is become something very very
important for us to pay attention to to
figure out how we can manage this
transition well so this is to me sort of
one of the main things that justifies or
what makes sort of this field of AI
governance important during this talk
I'll try to show roughly four things
that will sort of continue to like
bolster this claim and sort of show why
I and my colleagues believe that this
particular field is very important to be
focusing on the first thing is sort of
this that I've already mentioned and
that sort of hopefully is something that
you all already have a sense of and so I
won't talk more about that the second is
that sort of shaping the development and
the use of this technology is
particularly important I'll give you
some some sort of reasons to believe
that as well
thirdly sort of there's a bunch of
things already happening in this space
there's a lot of interest in artificial
intelligence in general and this means
that the governance structures are the
structures that kind of that are going
to
how this technology is used and how it's
developed over the coming decades
they're being set right now and then the
fourth claim is that sort of the
thinking the our understanding of how we
want to govern this technology is really
lagging behind and so that really gives
you a big impetus gives you a big reason
to believe that we should be paying a
lot of attention to and we should be
putting resources into thinking
carefully about how we want to manage
this transition into a world where we
have systems that are as capable as
maybe like a human level machine
intelligence so for the rest of this
talk I'll firstly give you this sort of
case of why it might be important to
shape the trajectory and the development
of this technology going forward
secondly I'll sort of explain what AI
governance is what we mean by this
thirdly I'll sort of bolster this claim
I'll suggest this claim that there's a
bunch of things happening in this space
already and then fourthly I'll
illustrate this last point that the sort
of thinking is lagging behind by
highlighting some of the research that
we're currently doing at the Center for
the governance of AI so starting with
sort of the potential reasons why you
might want to to shape the development
of this technology positively this is
the first claim that a lot of people
will make often when they when they give
talks of this nature is they'll say
something like artificial intelligence
is likely to be sort of the new general
purpose technology and so people will
say things like artificial intelligence
is like CBS impactful ass electricity
for example and then itself I think
gives somewhat good reason to believe
that it's important to sort of shape the
development of this technology
positively if that's this is the
technology that is going to sort of
affect the world in all kinds of ways
going forward but I think we can also
get more specific and one way to do so
is to sort of highlight what might be
some of those sort of potential risks
from this technology I think one useful
way to do this is to sort of carve up
the space in these three different
buckets so the first bucket is this
bucket of accidents so basically we
might imagine that we sort of we develop
systems we
try to have them do something useful but
something happens there's an accident
and this could lead to some specific
risk we already see sort of small
versions of this today so for example we
have a bunch of cases where you are
trying to sort of make a system play a
game for example and it makes kinds of
silly mistakes so one example the
example I have up here is from open AI a
few years ago when they were they were
building a system that was going to play
this game called coal strainer where
you're you're you know you're driving a
boat and this boat is supposed to win a
race and if you win the race you get a
bunch of points and so what do they do
to figure out how to sort of how to
specify the reward function of this
system they basically just tell the
system to maximize its points seems good
seem sensible what happens well the
system finds that there might be a
better unexpected way to reach its goal
to maximize its reward function and so
what it does is it finds these green
little dots perhaps you can see them see
them up here these little things and it
turns out if you collect these dots you
get like a small number of points you
get like a hundred points and so the
system will find these find these sort
of little green dots get some points and
then what happens is that these dots
actually replenish and they do so
indefinitely and so what the system ends
up doing is just collecting these dots
over and over and over again and so it
turns out that it's found sort of the
best way to solve the problem which is
to maximize its reward but it does so in
a way that we really weren't expecting
and so sort of this this case might seem
sort of laughable this case might be a
case where you feel like okay well we'll
just sort of it doesn't really matter
however when we start sort of making
sure that these systems these machine
learning systems are used in more and
more sort of important situations when
we start having systems that manage sort
of our electricity grids for example
then these kinds of problems these kinds
of accidents won't be a laughing matter
anymore they'll be very very important
and so these are the kinds of things
that we need to be looking at for going
forward and will need to find solutions
to how to avoid these kinds of accident
the second category is this category of
malicious use so the idea is roughly if
artificial intelligence is going to be a
powerful technology if it's gonna be
useful for a whole range of things then
plausibly it will also be useful for a
whole range of things that sort of
malicious actors will want to do that
bad guys will want to do and so we have
sort of a number of examples will be
we've sort of mentioned in this in this
general case one example it might be you
might think that sort of authoritarian
regimes might be able to use this this
technology you might be able to use it
for for surveillance people discuss that
maybe it might be used for fake news
those kinds of things so that's sort of
one category of other other kinds of
risks we might see going forward and
then the third category that I died
often think about is sort of this
category of structure and so the idea
here is that it's not that we have any
particular actor that's trying to do
something bad and it's not that they
sort of accidentally build a system that
does something that we didn't want it to
do but rather what happens is that sort
of the when the new technology has
developed the incentive structures the
sort of landscape of sort of my
incentives will change and this will
have negative consequences and so sort
of one example here from from sort of
history might be might be this this here
which is a Gatling gun so this is sort
of an early model of the machine gun and
so dr. gatlin who was the person who
invented the Gatling gun he was actually
pretty happy when he when he developed
this technology developed this gun and
the 1800s because he believed that this
might be a way to sort of decrease
deaths in Wars because it just means
that you can shoot the same number of
bullets you just need to have fewer
soldiers doing it and so his thought was
okay so what will happen is we'll just
have fewer soldiers on the field and
things will be great
however as as we all know this is not
what happened what happened was that we
had the same number of soldiers or maybe
even more soldiers on the field we just
had a whole bunch more bullets and this
technology along with other technologies
such as like barbed wire for example
meant that during World War one we ended
up in a state where sort of if you were
defending it was sort of very very
advantageous to you it was very easy to
defend much more so in than in previous
worse and since we sort of the people
thinking about war etc they didn't sort
of update their models they didn't
update their thinking about how war
works quickly enough this led to a huge
number of under Ness unnecessary deaths
and so this is sort of just an
illustrative example of what might
happen when you have a new technology it
changes the landscape and if we don't
sort of keep up with this how this
landscape changes bad things might
happen in artificial intelligence I
think there's a number of cases that
might sort of go into this bucket of
structure I think one one sort of set of
problems that might end up here might be
things like if you think that artificial
intelligence might sort of lead to maybe
increased inequality because you might
have sort of a concentration of wealth
those kinds of things might might sort
of fall into this bucket and beyond this
there's a whole number of sort of other
challenges that you might want to think
of a bunch of them you will have heard
of before I won't go through all of
these
I should also probably mention that a
bunch of these tech not like certif
risks or sort of might also sort of take
the might be that we should have miss
out on benefits as well so a lot of the
things I've said so far might sound like
I think that in general there's just a
whole bunch of risks that we need to
avoid the main reason for this is not
because I think that there's a sort of
much broader space of risks than
benefits it's rather that the risks are
probably the things that we particularly
want to manage whereas the benefits are
sort of more likely to happen by
themselves is sort of the the general
thinking so this is give you some reason
to believe that sort of shaping the sort
of development the trajectory of this
technology of artificial intelligence
and machine learning of this whole set
of technologies seems to be important so
let's move on to the next bit which is
why a governance because of this might
be important and sort of what what that
might be it has sort of two parts the
first part is sort of a descriptive part
so this is where we're trying to figure
out
we're trying to figure out sort of what
actually shapes behavior in this space
and so the basic question is sort of
what is the context and what are sort of
the institutions that shaped the
behavior of the important actors in this
space so sort of what shapes the choices
of sort of researchers what they spend
their time on what companies do what
governments do in this space and you can
sort of contrast this to other ways to
try to find out sort of how how
technology might affect the world which
is sort of like technical solutions so
there's a bunch of work being done on
sort of how do we build systems that
sort of don't have accidents how do we
build systems that maybe aren't able to
be used by malicious actors for example
but here we're thinking about sort of
the political context Assoc all the
logical context for example that will
shape people's behavior another sort of
misunderstanding people often have when
we talk about governance is that people
sometimes think that you when you use
this word governance that you're talking
about governments in particular and
you're talking about them sort of
constituting laws but we're taking sort
of a much broader perspective so for
example where we're looking at other
actors as well a lot of the other actors
in this space when it comes to the
development of AI will likely be like at
least as important as as government's at
least for the coming few years so for
examples of the top labs researchers the
public and these kinds of things so we
take that sort of broader perspective in
that sense and then we also want to look
at sort of other mechanisms there's a
whole bunch of other things you can do
to try to shape sort of how a technology
develops then just sort of laws and I
think we should have that much much as a
broader perspective and think about
other ways to incentivize actors
companies etc to to do what we we would
like them to do and then sort of the
last bit is that we take a our
governance to be something normative I
would take it to be about how not just
how the world is or how it's likely to
develop but also how we want it to
develop and so the reason that we're
doing this research is because we
believe it's important to shape the
development of this technology in a
positive direction and so we want to
figure out what should we do how should
this technology develop going forward
and sort of another sort of last point
to make on this sort of AI governance
issue or sort of what what this is is
that it's likely to be a very very
difficult challenge it's a very sort of
big task we've set ourselves and it's
likely to be be quite tricky because of
a number of reasons in the main sort of
the main reason that it might be very
tricky is because of this thing I
mentioned earlier that people sometimes
think of this as a general-purpose
technology and if it is if it can be
used in a whole range of things if it's
similar to electricity then you might
imagine that it's sort of quite tricky
to try it quite tricky to govern because
it's going to be very very diffuse and
it's gonna come with a number of the
benefits that we want to capture but it
might also come with some some risks and
if that's the case it's gonna be very
very tricky to govern so you have some
sensible AR governances you have some
sense of sort of the kinds of challenges
that we're trying to try and tackle I'll
now give you like a brief sense of what
is sort of actually happening in this
space right now the first thing to to
notice is that sort of governments are
paying a lot of attention to this over
the past few years we've had at least 30
countries and it's sort of racking up
more day-by-day who are sort of devising
their own AI national strategies until
Sweden's done this which is sort of
partly led to the sort of I guess
they're like boom of AI stuff going on
in Gothenburg and along with a whole
host of other countries they've sort of
over the past few years they've started
to do these national strategies instead
of another sort of tag on effect that's
happening right now is that most of
these countries are then also starting
to think about sort of how they would
like to govern the technology or sort of
ethical principles that should shape the
technology and those kinds of things and
so there's a whole lot of attention
being paid to this this particular issue
among governments and then on top of
this sort of companies are also starting
to pay attention they also started to
pay attention to sort of AI governance
but also AI FX and are doing a whole
bunch of things to to sort of make sure
that sort of these that are sort of
shaping the systems that we'll have
going forward in how this technology is
being is being developed
and so some examples here one one sort
of sort of the oldest example is the
Asilomar principles that were decided
sort of agreed upon in 2017 which was
when you had sort of the very very top
top minds and top researchers and
companies in artificial intelligence
gathered in Puerto Rico and basically
just sort of came together to define a
set of principles about sort of what do
we want out of artificial intelligence
and said all kinds of things such as we
want artificial intelligence to benefit
humanity and we should develop this
technology for for sort of the good of
all and we should make sure that the
benefits are sort of widely distributed
so this is sort of a very positive
interesting development on top of this
were also starting to see things like AI
companies are sort of starting to pay
attention to this issue as well
so for example deep mind one of the
leaders in this field have a sort of
internal team whose purpose is to think
about AI FX AI policy etc another
interesting example is open AI in in San
Francisco who are also starting to think
about about these these issues very very
seriously I would recommend you have a
look if you're inserting these things
that would recommend you have a look at
their open a charter where they sort of
outlined what their principles are in
terms of how they will they will deal
with questions such as sort of what do
we do in a world of artificial sort of
transformative artificial intelligence
and they'll say things like they will
make sure that if they sort of come to
be very very sort of they come to
develop some very very useful in
powerful technology they'll make sure
that is sort of widely shared and then
sort of a last thing to mention is the
partnership on AI which is sort of a
conglomeration of companies AI companies
basically of the world they're sort of
the top companies where they are sort of
starting to discuss these kinds of
issues and starting to sort of organize
themselves around what kinds of
principles they would like this
technology to be to be following going
for it so ensure there's a whole bunch
of things going on and now I'll try to
sort of give you a sense of how there's
a whole bunch of sort of questions that
we would like to have answered in order
to sort of decide how we want this AI
governance system to be set up that
don't currently have an answer to the
way I'll illustrate this is just to give
some examples of research that we're
currently doing at the Center for the
governance of AI this is roughly what we
look like here on the left you have our
team and most of the research most of
the work we do is just doing research on
these questions of that I'll sort of
point to a bit later and then we also do
quite a bit of outreach about this
research as well to make sure that not
just that we're sort of thinking about
these things but we're also sort of
spreading the knowledge that we
hopefully happen to happen to create so
when we do this kind of work
usually we divided up into instead of
this general structure so if you're if
you're really interested in in this
space if you think that the questions
that I'm talking about now are really
interesting I would really recommend
that you have a look at our research
agenda over here which was which
published last year where we sort of
outline what are the main questions that
we need to answer in in the field of AI
governance we divide that agenda up into
four bits the first bit is this bit
called technical landscape so here we're
trying to figure out how will Vista
Technology develop going forward in the
coming few decades and maybe in the
coming sort of century secondly we're
trying to figure out sort of the
politics and sort of international
security of artificial intelligence and
so here we're trying to understand okay
so the technology will be sort of
developed in this kind of way we'll have
these kinds of capabilities well how is
that going to affect the world how is
that going to affect things like mass
unemployment how is that going to affect
things like healthcare etc thirdly we
want to figure out sort of ideal
governance and so here we're trying to
answer the question well what kinds of
sort of governance systems would we like
to see and then fourthly we try to
translate all these sort of three
previous questions the answers are those
kinds of questions into actual sort of
next steps if today we are asked by a
government or by a company well what
should we do what do we tell them that's
what we're trying to do out of this sort
of fourth fourth area that we call
policy I'll go through some some sort of
brief examples of some of our some of
our
research in terms of the sort map in the
technical landscape some of the things
that we've done one thing is sort of one
of our one of our directors Nick Bostrom
wrote this book called super
intelligence which was sort of one of
the things that sort of started to spark
debates and issues around AI safety a
couple of years back we're also
currently doing a lot of research when
it comes to artificial intelligence in
China and so Jeff ting at our Center
does a lot of work about sort of what
are the capabilities what is happening
when it comes to artificial intelligence
in China both understanding what's
happening on this sort of the private
firm side but also understanding what
the Chinese government is doing in this
space and here we're sort of interested
both in sort of what are those actors
doing but we're also interested in
questions such as sort of how how are
the sort of the capabilities of Chinese
researchers or Chinese companies how are
they compared to the rest of the world
also seems like a very important
question to figure out we also have this
work of sort of expert service that I
mentioned mentioned previously we're
going to be sort of continuing to do
this work and I'm particularly instant
in us to trying to figure out two things
when we do that work one question is
just how accurate how useful are the
predictions of these machine learning
researchers it might be the case that
they're actually not very very useful or
very good at forecasting the development
of their of their field that's something
I'd like us to to figure out a lot more
about and then the second thing is that
we'll probably do a lot of work to sort
of understand more the views of these
researchers on a whole host of other
questions so questions such as sort of
how they view the the potential effects
of their technology etc
when it comes to thinking about politics
and security we also want to look at
sort of historical case studies one one
sort of set of case studies that we're
instead in our questions to do with how
previously powerful technologies or
technologies that happen to be powerful
to militaries or nation-states how have
they been handled in the past and so
here you might be interested in for
example how nuclear weapons were handled
and so for example we're instead in
looking at something called the Baruch
plan we're sort of right after the
Second World War
the US had the nuclear bomb Soviet the
Soviets didn't yet and there were some
debates about how to figure out where
people were trying to figure out how we
could make sure that nuclear weapons
didn't proliferate that more countries
didn't sort of get access to the bomb
and they were actually some debates in
the UN discussions about actually
setting up systems where where that
wouldn't proliferate more beyond sort of
the u.s. or maybe even the US would sort
of decommission their bombs as well and
so we're instead in finding outs in
those particular scenarios sort of how
did we as a as sort of an international
community handle those kinds of
technologies and might be there be
things there for us to learn if we're
going to be applying artificial
intelligence technologies to to the
military for example we're also
interested in in sort of a set of
questions that we call sort of levers of
influence where we're trying to
understand sort of what are the ways in
which the important actors in the space
for example firms and governments are
going to interact going forward what are
sort of the levers they can pull to
influence each other going forward and
so for example you might be interested
in how is the u.s. going to use
immigration policy or policies around
export controls to affect the
international community of machine
learning or artificial intelligence
we've also done some work for example
for example this report looking at sort
of mapping up the whole space of what
might be some sort of malicious
potential military malicious uses of
artificial intelligence in terms of
ideal governance we are interested in a
whole bunch of questions one set of
questions that you might be interested
in here might be how the public what the
public views are and this is some work
that we're going to do a lot more of
going forward here you might be
interested in questions such as what are
the challenges that art if that the
public believes are the greatest when it
comes to artificial intelligence we
recently published a report about
American attitudes towards artificial
intelligence and we're going to be doing
this with the EU and with China as well
going forward I'll highlight like two
results that I found particularly
interesting on the left here you have
results from us
asking participants in the survey about
their trust in various sort of different
actors so we asked them about how much
do they trust for example governments or
maybe companies to develop and to manage
artificial intelligence here you might
sort of you see a number of different
things one thing that's interesting is
that the US military sticks out as
something that the US public is sort of
particularly trusting in also they're
particularly trusting in university
researchers then you have this sort of
general cluster of sort of government
bodies and sort of tech companies and
here on the very very Left we have
Facebook who are sort of the least the
least trusted actor in this space and
interestingly this was even sort of
before the Cambridge analytical scandal
and so I don't know where they'll be
today
and then another sort of set of results
that I found interesting was we asked
them a similar question to this what we
did in this expert survey we tried to
understand what do does the public
believe about the development of
artificial intelligence how quickly will
they think that the developer will go
there's a this is a slightly different
question where the bar of HL mi or
high-level machine intelligence was
slightly lower than in the expert survey
but anyways we see something really
interesting which is that in general the
the predictions of the public are a lot
higher and so they put sort of a 50%
chance of getting to high-level machine
intelligence where in layman's terms
roughly the system is as good or systems
are as capable as humans at 50% in sort
of roughly ten years which was sort of a
very surprising number to me and then
we're also starting to think about sort
of policy we're starting to think about
what we want actors to do today in this
space one example of this is that we're
doing this work we call AI for social
responsibility or the cooperation Metta
norm or a I race to the top we haven't
quite settled our name quite yet and
here we're trying to understand what
does it mean for an actor for example a
firm to be sort of a good a good citizen
in the space to be sort of pro-social
and then we're trying to understand how
can we affect them or sort of
how can we incentivize them to to sort
of act in pro-social or good ways so
basically how can you make sure that
it's sort of pace to be nice and then
another bit of work that we're doing is
about something called the windfall
clause the general idea is that you as a
company you sign up to a contract
basically do you commit yourself to if
you become really really big if you sort
of win the AI lottery or something such
that you just have like a huge part of
the global market then you will commit
to give back a big portion of that money
too so the global Commons to give that
back to to humanity and so on sort of
the small chance that that might happen
we can then see that sort of benefits
would be distributed more widely and
we've sort of we're looking instead of
there whether that's legal and sort of
if that's legal how it might we go about
making sure that companies would would
actually sign up to something like this
so in summary hopefully I've I've
convinced you of some of these things
and in general I'm interested in sort of
this this last point that the thinking
seems to be lagging behind there's a
whole host of really important questions
that we currently don't have very good
answers to that I would like us to
really put sort of put more energy into
and one way to do so is to work with us
or to sort of get involved in this space
and in general I think this this is sort
of some general tips of what you might
do a sort of a technologist I would
suggest that you sort of educate
yourself about these kinds of issues
about AI ethics about AI governance I
would also suggest that you if you're
interested my going to doing research in
this space in particular things that I
be excited about would be either the
sort of technical safety issues so you
might want to be thinking about things
like how do we build robust systems but
also issues to do with AR forecasting
and then lastly I think a really really
important role of people such as
yourselves is making sure that decision
makers such that like powerful actors in
this space are held accountable so learn
more about these kinds of issues and
sort of pay attention to what the big
sort of companies for example do in this
space and make sure that they're held
accountable to making sure that this
technology is used to the benefit of
sort of us all thank you all very much
[Applause]
Oh so if anyone has any questions please
feel free if no one else has any
questions yet Oh Tanya I was curious
about the prediction or sort of the the
trust you put in these predictions so I
guess who qualifies to be a machine
learning expert in this context and and
sort of can you see any differences
between what their role is in the
ecosystem because I imagine if you're
like machine they're learning DevOps
engineer you're not really Ferring
fearing terminators so much yeah so we
so the way we did this survey was that
we went to the sort of the top
conferences in in sort of the academic
conferences in in machine learning
so ICML and nips
now nerves and we we sort of yeah we
surveyed those respondents so they would
be I guess would tend towards them being
more the academic side of things also
interestingly in this survey when we
looked at when we surveyed the public we
also looked at whether they had machine
learning knowledge tour experience or if
they were software developers for
example and we found that those people
would have in general would have longer
timelines they would believe that
development would go slower then sort of
people had less expertise believed in
general I find it really difficult to
know how much to trust it I think that
we don't have a lot of good information
about something like this this is
probably one of the most difficult
things to forecast and I think yeah
maybe this is this is sort of among the
best evidence we have but it's not very
good at all question here in the front
thank you very much for an interesting
speech I was wondering I hear sometimes
that Europe is foking focusing a lot
more and ethics than they do in the
United States and do you think this is
true and do you think you can see
differences within Europe or yeah just
in general is this kind of a local issue
that we are talking about only in Europe
right now so it definitely seems true if
you talk about governments so in general
the US government over the past few
years hasn't been done doing a lot in
the space of sort of AI in general and
especially not when it comes to sort of
AI governance or ethics I'm unsure
whether that's sort of about the current
current regime we have over there or if
it's if it's something more systemic if
you look more at sort of what other
parts of the system we're doing so if
you look at what companies are doing and
sort of what the public is doing I don't
have a I wouldn't be confident in saying
that there's like more discussion in the
EU I think there's a whole bunch of
things going on in the u.s. one I guess
like one example is just the recent what
recently happened with Google who like
recently set up this AI Ethics Board and
they had to take it down or chose to
take it down within a week to disband
this board after getting some really
really strong backlash from from the
public and also from sort of researchers
internally because of the the people
they chose to have on their how on the
council so I think there's there's also
a huge amount of discussion about these
issues in the US as well any more
questions
then maybe I can have a question using
laws we have a built-in enforcement
system we know how to punish people who
break laws but do you believe that we as
a community will self govern and will
self enforce these ideas will seems I
don't really know will seems like a
strong word well I definitely know so I
think I think there's some reasons to
believe that sort of the the machine
learning community is sort of slightly
different from from other kinds of
communities or like I think we're like
quite lucky and instead of the community
that happens to be developing this
technology
in that it's sort of you have some kind
of like bits of hacker culture are a
part of sort of the machine learning
cultures where like you want to do
things that are good for the world and
and those kinds of things which i think
is really really important and so I
think I think there's definitely some
there's definitely hope there and I
think that we can we can sort of improve
the self governance of this of this
ecosystem really well by for example
like us in this room taking
responsibility for it but I think that
you should also like going forward as we
learn more about what the impacts of
artificial intelligence will be I think
that we will probably need more and more
sort of governments to be involved or so
like actual policymaking as well so if
there are no we have some time for a
short question in the middle to the
right
Thanks yeah just a question how how
international are you because I'm
getting the sort of gist that this is
this something we do in Europe in the
Western world and so far but I'm
thinking about China Russia and the rest
of the the world where AI is developing
rapidly and you know China has hasn't
the same set of laws as we do here so
how are you progressing internationally
do you mean sort of us specifically as a
research group right
okay yeah so we we try to do the
research that we think is going to be
the most impactful and important in this
space and if so then China is going to
be like one of the most important sort
of actors to be to be thinking about
along with sort of the US for example
and so we do a but a whole bunch of
research when it comes to when it comes
to the developments going on in China as
well so for example Jeff Danang is one
of our researchers who hope it's a big
focus on that and then we have some some
other people are affiliated with us who
work specifically on AI users in China
all right thank you so a big thanks to
Marcus
[Applause]


What is AI safety? | ZDNet

is our AI future more positive than even
the most ambitious sci-fi riders imagine
I'm Tanya Hall for ZDNet and tech
Republic and joining me is max tegmark
MIT professor and author of live 3.0
welcome max what is the mission of the
future of life Institute you simply want
the future of life to exist and be as
inspiring as possible and I'm optimistic
that we can create a really inspiring
high-tech future but only if we win the
race between the growing power of our
technology and the growing wisdom with
which we manage this technology and the
challenge here is to switch strategies
because we used to win this wisdom race
by our learning for mistakes you know
first we minted fire screwed up a bunch
of times and then we invented the fire
extinguisher but with more powerful
technology like nuclear weapons and
future superhuman artificial
intelligence we don't want to learn from
mistakes it's much better to plan ahead
and get things right the first time
that's our mission what is artificial
intelligence safety and why should we be
researching it AI safety is simply that
wisdom we need to make sure that our AI
systems are not just powerful but that
they actually benefit us do what we want
them to do you know we're putting AI in
charge of evermore infrastructure and
decisions that affect people's lives so
we have to transform today's buggy and
hackable computers into robust AI
systems that we can really trust because
otherwise all this shiny new technology
we're making can malfunction and harm us
or get hacked and be turned against us
when discussing how advanced AI can
become dangerous you say it it isn't
malice but its competence explain that
that's exactly right Hollywood makes us
worry about the wrong things of AI sort
of turning evil but you know the reason
that we have more power on this planet
we humans than Tigers do isn't because
we're stronger but because
smarter so future super intelligent AI
will give great power to whoever has it
people or maybe even even itself so the
challenge is to make sure that that
competence that power is aligned with
what we want to happen it was fine for
it for you and me to be surrounded by
more intelligent beings when we were
little kids cuz mommy and daddy had
goals be nice to us and help us flourish
right and that means as we make machines
more powerful now we have to not just
focus on making them smart and capable
but also educate them like good parents
make sure that they can understand our
goals adopt our goals and you know
retain our goals and those are her three
hard questions if you take your future
self-driving car to the airport and tell
it to get there as fast as possible you
arrive covered in vomit and chased by
helicopters and you were like no no no
that's not what I asked for it and it
said but exactly what you asked for
you've appreciated now how hard it is to
get machines to understand what you
really want right and also anyone who
has kids knows the difference between
getting our children to understand what
we want and for them to adopt these
goals and I should do what we want
that's gonna be at least as hard with
machines and finally we want to make
sure if machines get ever smarter that
they don't just get bored where their
goals are being nice to us you know the
main way my kids have gotten bored with
playing with Legos but did they retain
them so that we always end up knowing
that we're working with AI is working
for us and not against us what are some
of the common myths regarding artificial
intelligence and what is actually closer
to the truth one common myth is that
intelligence is something mysterious
that can only exist inside of biological
organisms like us where the fact is it's
all about information processing this is
what's giving us the whole AI revolution
right the insight that it doesn't matter
whether the information is processed by
a carbon atom
side of neurons and brains or by silicon
atoms inside of machines it's and if you
have machines that aren't limited by
what fits through mommy's birth canal
you know clearly we have the potential
to make much smarter things then our
selves one day which will either be I
think the best thing or the worst thing
ever to happen to humanity and I want to
work hard for the former disruptions
occur when previously unrelated
technologies confirm converge in
unanticipated ways
what kind of disruption might occur if
AI merges with biotechnology yeah this
is a great example of the need for this
sort of wisdom development if you look
at bad accidents that have happened with
technology in the past it was very
frequently just unanticipated
consequences not that samba are they or
something was evil but do we hadn´t
through carefully enough
and fortunately thinking about how these
unforeseen consequences has a long
tradition it's called safety engineering
at MIT where I worked some people
misunderstand this as Luddite
scaremongering and trying to freak
people out but safety engineering is
exactly why for example NASA
successfully put people on the moon
because they thought through all of
these things that could go wrong to make
sure it went right and this is exactly
what we need to do as a community now
with AI think through all the things
that go wrong to make sure they don't
and also I think important is to have a
more long-term vision for what we're
trying to accomplish here because if we
don't
and ask these questions we're just in
the process of trying to make ourselves
obsolete as fast as possible right first
we group made our muscles kind of lost
somewhat obsolete with the Industrial
Revolution which worked fine cuz we
educated ourselves and started working
over their brains and now with AI and
we're trying to make our brains obsolete
but surely we humans can be more
ambitious than that and envision truly
inspiring society where we flourished
and we trade all this well for the AI
and we share it so everyone gets better
off and we feel rather than redundant
and unnecessary we feel empowered by
this and having this awesome future but
this is a challenge and I often have
students walking into my office asking
for career advice and I always ask them
what vision they have for further future
and if all she can say is oh no maybe
I'll have cancer maybe I'll get murdered
terrible strategy for career planning
right but I feel we humans as a species
are making exactly that mistake every
time we go to the movies and watch
something about the future one dystopia
after other and a terminator Blade
Runner we need positive visions because
the more we can articulate a positive
vision that we're all excited about no
the more likely we are to get there I've
interviewed entrepreneurs and scientists
who seek to impart emotional
intelligence to AI to drive sales
products and services what are the
ethical issues around using AI to
influence human behavior through
emotions well there's obviously a lot of
ways in which you can just manipulate
people in doing things that aren't in
their own interest buying things they
don't need voting against their
interests we've already seen some extent
some stuff like this we Cambridge
analytical for example at the same time
there's also positive ways in which we
can make the ecology manipulate us that
sort of bring out the best in us we do
that every day when we have it your
alarm clock gets off goes off as
technology changing your behavior so you
don't miss that important meeting
and so on and ideally we can we can one
day make AI that is specifically
designed with us in control you know to
really help us be the person we want to
be rather than just manipulated into
something else you break down AI
artificial intelligence into three areas
power steering and destination explain
those yeah if you have any technology
that you want to be really ambitious
with it's not enough to just focus on
making it powerful right you would never
go into a kindergarten and say hey kids
here are these really powerful hand
grenades why don't you play with them
what could possibly go wrong right you
also have to think about how to steer
the technology to control it and where
you want to go with it that's what we do
when we send rockets to the moon and
beyond and that's I think metaphorically
what we need to do is we make AI more
powerful the steering of AI is all the
AI safety stuff we discussed how do you
make it not buggy how do you make it not
hackable and then the destination is the
question of what kind of society we're
hoping to create what sort of future do
we want to wake up in in 50 years if
machines can do all the job better and
cheaper than us and I I think that
there's been way too much emphasis in
the media and particularly on all the
risks genic is it's easier for us to
think about all the ways we can screw up
right and what most religions have much
more details on hell that I'm happen for
that reason but it's incredibly
important to just envision something
positive if and there is a lot to be
positive about everything I love about
intelligent about Society today is the
product of intelligence right so if we
can amplify our intelligence to cure all
diseases to figure out how to lift
everybody out of poverty and make
everybody free and have the opportunity
to really live out their dreams on earth
and if they want to elsewhere in the
cosmos now then life can really flourish
not just for the next election side
but for billions of years and there is
no technology but more power to make
these things happen than AI because
ultimately so it's so far all the
technology that we built all the
advances you made have been done with
our intelligence if we can make AI as
capable as us right it unlocks the
potential to develop all this other
technology to dramatically faster than
we have a dreamt of and even the most
ambitious scifi writers I think have
been too pessimistic actually in terms
of what's physically possible
interesting take max tegmark MIT
professor and offer author of live 3.0
if somebody maybe wants a copy of your
book I certainly have one or maybe they
want to connect with you or take one of
your classes how can they do that well
people want to have more insight into
where this is all going and how they can
make you work for them I hope they'll
find my book life's people know useful
because I wrote it exactly for
intelligent people who are just
interested in this without having a very
nerdy background make myself and if they
have questions after the mean book we
can email me tech market mit.edu perfect
thanks again so much for joining me and
if you guys want to find more of my
interviews you can do that right here on
ZDNet or TechRepublic
or go to my website tanya hall net
thanks for watching
[Music]


Possible Paths to Artificial General Intelligence

[Music]
all right thank you very much everyone
so I'm Josh and the other panels are
already introduced I'm going to start
off with just very brief introductory
remarks and then we're each gonna give
short opening statements I'm not sure
how much debate is gonna because I think
actually there's a really interesting
remarkable agreement among many of the
panelists in the broad sense about the
kind of path Iran but what I'd like you
guys to say in your introductory
statement would like you to focus on is
what path I mean so first of all it's a
above the five of us here I'm sure you
know something about each of us but I'd
say or out of the five of us are
actually working on AI you know as one
of our main things Nick has been more of
a thinker about about a broad range of
paths and their implications and so I
want to start off by asking the four the
four people here and I'll go at the end
and say a little bit about my view what
half a GI are you on
yoshua actually just give a really great
 explain that so you could you could
summarize it or just a that's right it
was
but what path are you on which means
like what's your goal and what's your
roadmap why did you choose that path and
how far along on it do you think you are
or you're part of the community is and
then we'll dig into that a little bit
and think about next step so um Arina do
you want to so I think we all agree that
the there are multiple viable paths to
enjoy and all of them come with the
certain trade-offs in terms of how long
it will take us to get to HR you could
take a particular path and what the
resulting HDI might look like in terms
of things like memory or computational
requirements so when I was making a
choice of which path you the question
asked myself is what is it that I want
each eye to actually look like and for
me I I want H I to do as well as humans
on the National tasks in our natural
world and so for me it's probably
natural to look for inspiration
and of course in our world we know that
there is this numerous examples of
biological intelligence each one evolved
for you can perform optimally in the
tiniest ways of the universe that their
body allows them to explore and we know
that these intelligences are very
diverse range from the distributed brain
of the centralized brain of the
vertebrates but also they share quite a
lot of similarities so for example if
you look at the human brain then we know
that cerebellum is shared with the
ancient ancestors like fish and parts of
the limbic system are shared with
lizards also if you even look at the
different areas of the cortex in a
single brain
it's widely agreed that they share a lot
of computational principles and what
architecture even though they might be
doing very different roles like visual
cortex person or district or sex
so it seems like there are some core
computational principles that are shared
between many examples of the existing
biological intelligence that we still
don't really understand so for me this
is the past try to figure out what are
these principles and I'm not talking
about trying to replicate the my new
details of like spiking dynamics kind of
works but what I care about are the kind
of core computational principles so if
you know of Mars way like levels of
description is more of the algorithmic
side things another thing that I think
is really important is that intelligence
does not exist in in the vacuum right if
we had a completely random world then we
would not have so there is something
about the structure of our universe in
the mouth of our world that enables
intelligence to exist so for me it's
really important to examine these
embodied interaction between the brain
and the world to understand where
intelligence comes from so this is my
approach trying to look both at the
structure of the world and the kind for
computational principles
of the brain to figure out what are
these the principles we can then report
to the artificial substrate and also I
think these paths is quite cool because
even if we don't reach HDI hopefully on
the way we can discover quite a lot of
interesting things about ourselves hey
great
go ahead hi my name's E and from Chinese
Academy of Science I'm working on brains
by the approach for artificial general
intelligence so I guess our that one is
quite similar but I do see well I do see
the difference actually well just know
we mentioned that what do we really need
for for the realization of artificial
intelligence do we need spiking patents
activities like that
I understand to my mind I would say that
science is in the detail so my approach
is more on the bringing is part it's
back in your network for for realize
artificial general intelligence so when
we talk to people well people say that
okay you're working on brains by the eye
that means your connection is - and then
I would answer this question like this
so has a connection astre do you reject
a symbolic AI and people always say yes
we don't really need symbolic AI but my
understanding is but is that for brains
by the AI what we need is not only it's
not only the connectors it's not only
the about a computational model and and
their self-organization of the network
but also how can the neural dynamics
give rise to symbolic reasoning so this
is truly important so in this case I
wouldn't say brains by AI is
connectionist so I'm not the first one
to say that like like in 1955 when
people have Western joint conference on
computer science there's a session a
year before the year before the AI first
conference in that Mo's say
water Pete says that emulating the
nervous system and emo and actually
emulating the fight the hierarchy of
final course is traditionally called the
mind will actually come to the same
thing there's no doubt so so in this
case what I expect is that for brain
inspired AI we need this energy and
collective efforts for both
connectionist and symbolic AI but more
in the details I would say so all my
work is based on single single neuron
modeling of different type of neuro and
how it gave rise to different kind of
functions and how this cognitive
function self or self organize together
to realize more of complex multitasking
and even AGI and for more for a more
specific level of detail so we're
working on self recognition different
levels of self-consciousness and fear of
mine so that I think this is a
fundamental use for Safeway I cannot
expect the world that an AGI system
without a sense of self that can really
understand the world and interact with
us so I think the building block should
be starting with very nice show stage of
self consciousness all the way down to
fear of mind cognitive empathy and also
emotional empathy and then we could have
the real safe AI okay great I'll just
say for myself again I think you know
again you saw your show statement we're
all inspired by cognitive science and
neuroscience me you know I come out of
cognitive science and my fundamental
goal is to reverse-engineer intelligence
in the human mind and that means also
the brain but I'm approaching more from
the functional behavioral and I'd say
software level of algorithms as opposed
to starting more from the network
structures and me right now what's
what's what's teams have exciting the
path but I think I'm on is what is in
some ways you know AI is oldest dream
about how to get two qubit intelligence
going back to Turing's paper in which he
proposed the Turing test
right the dream of being able to build a
machine that grows into intelligence the
way a human being does that starts like
a baby and learns like a child that you
know Turing proposed that famously as
the only ID he had about how you could
do it and it's been it's been promoted
by you know out these early AI
conferences by Marvin Minsky John
McCarthy all in some sense all of the
great AI thought leaders from the
beginning and you know Yoshio and I
agree you know very much like we show
the same videos you know the idea of
like what is that basic common sense
that even those young kids have when
they're playing with blocks or cups
right I think if there's a good reason
for this as as a giggling root AI
because it's actually the only known
scaling route in the known universe that
we know actually works right a human
child is the only learning system that
reliably reproducibly demonstrable grows
into adult human level intelligence
starting from monthly so if we could
reverse engineer how that works then you
know that would that would be a route
and it's also a route that we can
measure there are milestones and metrics
and there's an entire field of cognitive
development that has for the last couple
of decades made great strides towards
actually laying out what that roadmap
might look like I'm always asking myself
also okay well if that's what we're
working on and all these great people
have thought about it why why hasn't it
worked before and why is it why might it
work now well I I think we're at a very
important time which is that the fields
of engineering and the fields of cosmic
development have both matured to the
point where they can talk to each other
where engineering just as yoshua said
we're engineering now and offer useful
tools to the people who are trying to
reverse-engineer how babies Minds work
right and where where the field of
developmental psychology hasn't has got
enough qualitative and even quantitative
insight that it can actually start to
guide those goals and hopefully they'll
be mature enough to talk to each other
and in the work I've been doing as a
cognitive scientist for the last 10
years or so that's exactly been doing
we've been building computational models
of instance perception infant common
sense and the learning mechanisms which
take that forward well I mean I'm
excited especially right now about what
I what I see is likely sort of a
moonshot stage for us is roughly the
stage of 18 months
old common sense the common sense that
is that is the height of where we get to
before language comes into the picture
but which language builds on is I think
that's where we have enough
understanding on the stein side and
enough maturity and our engineering
toolkit that we can really you know get
there in a in a practical way we're just
at the beginning of that because then
you'd have to ask well what happens when
language comes into the picture and
culture I completely agree again agree
with yoshua of the sensuality of culture
and the cognitive mechanisms of language
that an end and empathy and other kinds
of things that support that as getting
us you know that's the real singularity
for human intelligence but I think we
understand enough about that first step
that that it's extremely exciting to be
working on that as a computational
cognitive scientist and as an AI
engineer okay so that's so that's so
that's that's the range of views which
in some sense we're all biologically
inspired in some ways of course there
are other possible paths to AGI and
again nick has has had a lot to say
about the different um let's say factors
there do you want to comment on on how
you see the panelists here thinking or
other possible paths I mean I I agree
this looks like kind of the model most
promising I mean may be fun just to map
out things if one sort of indexes say on
unbend your intuitions and vision and
then once these what I could one diverge
from that in different directions and
get different sort of ambitions of this
so I think one one direction would be to
say we need even more inspiration
probality like we really need to capture
the nitty gritty quality that the exact
dynamics of spiking trains and dendritic
compartments and and so forth that could
be a vision somebody would have said we
really focus more neuroscience then may
I and and at the extreme of that you
have people who think we will first get
the machine intelligence by not just
being inspired by the brain but by
replicating the whole thing with some
kind of old brain emulation right that's
what you're gonna work which I agree
looks much less likely but if one kind
of maps out the space of possibilities
that are championed by at least some you
know
non stupid people then that could exist
on that map then I could going in
another direction and say well we can
abstract more from from the brain part
and look more at the mind part maybe
good old-fashioned area looks bad but
there are other ways in which you could
go up a level of abstraction you could
think maybe some kind of evolutionary
algorithm like some algo scourge would
with enough compute to be the way we get
it so like do away with all of this
human scientists trying to figure out
the mechanisms and just hope we find it
in a beginner search base or maybe
something that you find the right kind
of idealization and it's something
different something more principles
something more based in your nature some
kind of graphs something put together by
repeated application of some simple
functions that you build up and and then
this this looks like a clue they do your
letter approach might looks like I kind
of dude you're way off approximating
what there's a cleaner
I guess if you take a more secure
district and say well hey what we need
to do is we are right now kind of
incapable and meet the first enhance
humans either collectively improve our
collective intelligence the scientific
community or individually with
biological enhancement that what would
be what face machine intolerance and so
I think this kind of maps had a little
space and one can maybe put different
people's approaches in there and I have
an impression that that you think
something between Benjo but with a
little bit more maybe theoretical
understanding of the modules and
specific things that human infants do
and then maybe kind of more cognitive
science yeah yeah I guess I I would say
well are you are you yeah no but and
yeah and you might think try to give me
more power and kind of similar in the
level of abstraction of inspiration that
we need yes so if we could talk about
our different levels of abstraction yeah
I think talking about different levels
of abstractions your interesting play
to discuss yeah and I want to comment on
makes point our evolutionary computation
so actually my view for intelligence is
not only about learning what your turn
is like milliseconds to days but also
for decades of development and for
billions of years of evolution yeah
so this is why in max book about like
3.0 saying that for AI we got an
opportunity to rewire both only both
software and hardware so let's say in
the biological brain what we have for
one concrete example is that we have
excitatory inhibitory neural so why not
50 and 50 half and a half is actually
not in the brain well actually in the
sensory cortex approximately 50 percent
of them is you know the mammalian brain
and when I am developing a spiking
neural network for for amnesty
recognition is actually I found that the
15% of the inhibitory is the optimal for
m-mister recognition so that means
through decades of evolution what we
found what the biology found is a
optimization procedure that helped us to
find the optimum to solve problems so in
this case now we got the power of
learning and development and also
evolution so that you can rewire the
whole system the whole system design and
not only for parameter to me for those
connections of the building blocks but
also to change the building blocks and
how they connect to each other so so
this is very important that you know the
scientific approach looks and says okay
there's adaptive process these you could
call them learning instant across all
these timescales write milliseconds
scale to millions of years of evolution
and with cultural evolution also in the
middle of that right and I think you
know that that might there might be
important differences of opinion there
right like you could say there's another
path right which may be referred to a
little bit that is very well represented
I think in some some parts of deep minds
are in an open ai for example and it's
often put out there as as what people
are may be scared about or excited about
is a kind of evolution the sort of
massively distributed deep RL as a model
of a synthetic model of evolution so
what we're going to
this evolutionary process with enough
games you know whether it's dota or
capture the flag or the entire open AI
gym universe and you know that I think
that's it that's it that's an
interesting thesis that that that
toolkit might be able to in some sense
get a kind of evolution but then there's
a question of can I do enough of the
like structural reek construction and
reconstruction that biological evolution
has done that you're pointing to and you
know that there there's more questions I
think um add a few things first of all
regarding the different paths that you
mentioned I think some are safer than
others I think you know this conference
is a lot about safety and in particular
I believe that if we more or less stick
to the human-like intelligence we end up
with something much safer than other
options because we kind of know the
limitations of human minds and they're
not infinite even if we as I said if we
have bigger minds and so what one thing
that's connected to this which i think
is also a good engineering reason for
going that way it's connected to what
karena was talking about when she talked
about the sort of shared principles
between intelligence in different
animals and presumably those principles
if we were to understand them would
allow us to build intelligent machines
now there's you know those you can think
of those principles like a small set of
laws like the laws of physics that would
be amazing I think it's like an amazing
hypothesis if this was true that we
could understand intelligence with such
a small set of principles then it would
have amazing consequences it would be of
course aesthetically pleasing but also
it would make it much easier to build
smaller machines which could be a good
thing or a bad thing but but it would be
mostly building a kind of intelligence
very close to human intelligence and the
other thing I want to mention is about
the levels of abstraction so the reason
I chose to do neural nets in the early
90s early 80s mid eighties was because I
felt that they were at the right level
of abstraction
sort of low-level notions of how like
particular neurons are actually
computing and high-level cognition in
the sense that with the the neural net
toolbox and especially the ones the one
we have today we can we can represent
high level cognitive computation and we
can also represent low level things like
one of the things I've been working on
is how could brain do back prop so low
level computation that gives rise to
credit assignment all of these things
can be talked about in that language so
that's that's that's very convenient one
thing I want to mention also about those
principles there's there's a lot of
effort in neuroscience and maybe to some
extent in cognitive science to describe
our human intelligence through all of
the things that we can do and in case of
neuroscience we like which neurons are
doing what sort of mapping out sort of
huge encyclopedia of every function that
the brain computes where it's done and
you know how things are connected to
each other I think this is hopeless
right it would take forever to
understand the brain that way if instead
we can figure out you know a small set
of principles that can give rise to this
computation then of course it would be
much simpler much faster to understand
the brain and and you know the principle
number one is learning right that that
you don't need to actually describe the
function computed by the brain you don't
need to need to describe which is like a
huge number of bits right you only need
to describe the learning procedure which
could be very very small number of bits
which could fit into your genome I think
I think I think I see the pre-linguistic
stage really important of course and
most animals are there yeah exactly and
I believe that the reason why children
have this amazing vocabulary spurns
because they already pre build the
concepts and now the fastest classes
just like fast attachment of poor labels
to these resistant concepts and I think
if we take AI know
human roots we have a danger of the eye
learning different kinds of concepts
than what and then it might actually be
harder for you to learn they like our
symbolic kind labels for for these
concepts which also means that then
humans might struggle to actually
communicate with AI so then we might
actually struggle to those things the
constrains rights with their
interpretability constraints that come
from that so I think that's kind of a
point I think I think that's a great
point
yeah I mean I think that if we want AI
in general whatever really form is going
on inside but if we want AI to live in a
human world to live safely effectively
valuably right to be able to be an
entity that humans can talk to teach and
trust the way we do with other people
even people we haven't met before then
in some sense right at least there's got
to be something like a human-like API
and at least in some in some sense just
as we have certain API to each other
right there better bit of justified by
some common mechanisms at some level I
think it's it's it's a root not only
both just both to safety but also to
some of the other dimensions of benefit
um maybe just to try to create some
controversy I don't know how much time
we have how much time should we take oh
okay okay great
um well okay so so so I can throw some
good traversable but let me just first
respond to what Nick said what you say
if that's okay and that'll try to create
good and then you can you can you can
hunter great controversy around that you
know like that controversy
so just to say yeah I mean I I think
there is one difference is I guess
coming coming as a cognitive scientist
maybe as opposed to a neuroscientist
there's a there are a few sort of
stylistic differences well I mean it's
also related to our orientation towards
the classical AI right I mean I you know
I think yo should thinks of classical AI
as well they had certain goals but it's
right right
mechanisms really I guess my you know my
feeling and I think that the sort of
collective insight of the field of
cognitive science is that actually
symbols are important there are really
symbols in the brain there are really
symbolic languages of abstractions
symbolic systems in the brain
and it's remarkable how that works and
we don't
know how it works but that there's
actually a lot of success we can have
with the combination of several ideas
rather than say in case of need you know
I wouldn't say neural networks are the
universal representation layer I would
say programs are the universal
representation those programs would be
differentiable programs of the sort that
a deep neural network is but they could
also be you know lists expressions they
can be grammars and I think to me what's
what's remarkable about the maturing
engineering toolkit is our understanding
of all the different possibilities of
programs as models and as different ways
to learn programs right if your models
in general are some kind of program then
learning has to be something like
programming and one way to program is to
is to you know it's the right code by
hand
another is to like as we've now to you
know found have a big differential
architecture with lots and lots of
parameters set up the right data set and
if it's end-to-end differentiable then
you can effectively tune and shape the
program using stochastic gradient
descent and such method but there are
many others but there's no programmer
that programs exactly but one of the
maybe one of the most exciting things in
machine learning right which has been
somewhat independent of deep learning
but is now also drawing on things from
deep learning is what's sometimes called
machine learning meets programming
languages or you know programs that
write programs so these are people
coming with a different technical
toolkit you know from the design
programming languages and program
synthesis and code that writes code it's
it's interestingly also somewhat related
to older ideas of genetic algorithms of
genetic programming but it's but it's
different from that it also draws on
some Bayesian tools but it's different
from that it draws on some techniques of
programming language design and even
spec means from compilers so I see that
area for example there's a hub that
match with evolution of intelligence in
animals and humans right yeah I mean I
think I think it matches very well right
I mean DNA is code right and a machine a
body and a brain is a machine that
builds itself from some code one and one
of the things that does is it writes new
programming languages both individually
and collectively so I think that that
general idea is actually a very powerful
one and and they're the many different
versions of it and correspond to the
different kind the different like scales
levels of analysis and scales of
adaptation
process these including learning
biological evolution cultural evolution
I mean that's a very big picture but I
think it's just that's one key
difference yeah yes so I totally agree
that programs are incredibly important
especially for human level intelligence
but I wonder what you think about this
idea that there are different levels of
intelligence abstract reasoning that
programs is kind of topside with that
but below that we have more kind of
grounded perception based abstractions
and I think there was an interesting
experiment by a little bear where he
kind of traveled around Russia at the
beginning of the 20th century asking
people kind of abstract reasoning
questions like you know giraffes don't
live in big cities Moscow is a big city
with maybe a draft there and people
could not answer this abstract question
they they were like well they were only
basing answers on their personal
experiences like I've never seen the
dress I can't answer this question so
and it seems like essentially this kind
of reasoning wasn't present in people
who didn't have formal education and it
seems like the formal education is what
gives people their bill that's one kind
of programs that's like the
consciousness prayer that's like
symbolic propositional programs but
there's lots of other programs in the
brain that I think we want to think
about how to make perception really work
right I mean in my view and grounded lie
mal I'm all for grounded language as
well but I think it brought excuse me a
broad view of programs that embraces the
strength of deep learning together with
the idea of generative models and you
know symbolic abstraction I think that's
also that's also the key to perception
and grounded language but you know
that's my view I think an another
interesting difference though is the
relation of the engineering enterprise
to the scientific enterprise yeah but
part of that is like how much scientific
detail matters does the whole connect
don't matter the spiking matter and so
on but the other is what's there what's
the relation between the experimental
world and sort of theoretical
engineering toolkit and a nice thing
about hogging the science is that you
know as opposed to the neuro approach is
that I can do those experiments in my
lap or on Mechanical Turk I can work
actively with people studying infants or
young children and to me that's
important it's important to say okay
when I say I want to build a machine
that starts like a baby and learns like
a child it's not simply nice my
intuition about how children think which
is you know Marvin Minsky as brilliant
as he was you know he was either stating
his
intuition or Piaget is intuition but
that you know Piaget was more or less a
contemporary of Turin a lot has been
discovered in the last couple of decades
and baby's brains it turns out start
they're not blank slates they start with
a lot of structure and learning is a lot
more sophisticated than and a lot more
diverse and a lot more symbolic we think
then simply you know understand and
that's the science the guy I have the
impression that what you're proposing is
to use the formalism which seemed
naturally adapted to system to
computation in order to explain system
one computation no I'm again I don't I
don't want they don't want don't I don't
want to miss communicate here right when
I say programs I don't simply mean less
programs right I mean a broad thing
which includes which includes deep
networks right I mean you know these
days in my group we do a lot of stuff
with deep networks and we and one of the
things we do is we use deep networks to
guide the construction of the symbolic
layer right and I think that's very
exciting um so I just to be just to beat
I want to be controversial in the right
ways but I don't want to miss
communicate I'm not saying oh we should
do what we should go back to symbolic a
I'm saying they had a good idea you guys
had a good idea to Judea pearl had a
good idea and and we have a maturing to
look at that allows us to see how these
several good ideas absolutely get
together and actually can extend each
other I mean I think that this embolic
is crucial but I've coming towards the
end rather than the beginning you have
to build up to it you have to sort of
earn your symbols by first sorting out
maybe on symbolic representation and
good old-fashioned EIS a little bit like
if you might in some very low-tech
primitive tribe this is like it just
plain shooting across the sky and I say
we want to build that so they're gonna
record we want this high-tech
civilization and have put together like
some wooden replica right it's never
gonna fly the way to actually get there
is the first invent simpler technologies
and then eventually our factories no
electricity and you can you can do the
whole thing but it's still helpful I
think when doing this upstream balling
stuff to know what you're aiming towards
because it
that you can have constraints in the
kinds of grounded representations you
crave if you know that in the end they
have to be combinatorial that's what I'm
proposing is that actually the the needs
of system to computation can be used to
help system structure right I think is
yes that would be a great idea because
you want it feel free to get you guide
the questions I love to how you guys on
the panel put yourself on the spectrum
of how brain inspired you wanted your
path to be from very little to very much
and I would love it if you can also
place yourself on a different axis in
terms of how intelligible you're
guessing that AGI is going to be where
one end of this expect would be to say
something like well it's just gonna be
ml all the way plus a bunch of clever
new tweaks but it's gonna be a
completely opaque black box will have a
very little clue of how it actually
works there's something maybe more
closer to what josh is saying where
maybe there is a lot of Goldfine sort of
married into that maybe a lot of Josh's
system too so that we can understand
more but but but there's gonna be both
right so if you look at humans and the
system one system to distinction there's
gonna be part of the explanation is
gonna be hard to express in words and as
a part of it which is higher level which
gives probably a good approximation of
the kind that humans communicate with
language and but that's not going to be
the complete explanation unlike what you
know go Phi was trying to explain
everything with with symbols to to
explain our actions or complicated
decisions there's going to be a part of
it that remains difficult to communicate
that's what I believe so I think my view
is that the what perception does in
biologic intelligence is exactly
grounding the symbols and then
everything even system one would then be
built on top of
so it's kind of disentangled
representations that the perception of
eels first and then you can do all sorts
of things on top of either model-based
planning or both for URL so even model 3
might be more in chapter 2 than just
black box but some some things remain
out of reach of interpretability think
about trying to explain the go moves of
alphago right maybe you can come up with
it was if it was trained with constraint
of having system to type of
representations at a high level you
might come up with some the same things
you find in a go book but it's not
enough to completely you know learn as
well I don't think I mean again I agree
with what you're saying on the other
hand explain ability is a key part of
human intelligence right the way if you
the way they seem complete right it's it
exactly I agree with that but we
shouldn't underweight its importance and
especially for a twin a human yeah right
when a human player learns go they learn
those explanations from the go book or a
master are crucial from the very
beginning and if we want a machine if
our goal and stay go or any other game
is to build a machine not that
ultimately with incredible amount of
compute learns to do this one thing at
superhuman level but can learn you know
more with the data complexity as you
said and the time complexity of a human
to learn so many different things so
quickly ok then I think explain ability
is going to be a key part of that I'm
both the input and output side and I
think again maybe we don't just agree on
that those things in terms of that's
like the endpoint of the path but in
terms of where the paths are in the near
term this this is where there is some
difference you know you got you're
talking about earning your symbols I
think in in certain ways that I that are
not fully appreciated in the deep
learning community we already have
earned our symbols there's already quite
remarkable things that we can do with
symbols and the deep learning people's
many of them are doing it anyway they
just don't they like tensorflow
or pi torch symbolic languages that
allow you to build that about individual
to build deep learning systems and that
allow collectively the cultural
evolution process of what you call the
deep learning community to innovate so
quickly in water up very much like
exponential it early exponential growth
counts and you can measure them in
people's
Google Scholar and and steered into
impact on the world okay that that's
those are cultural evolution processes
which of course we don't know how to
completely engineer those things in
artificial systems nearly at least those
symbolic Costas T's at the scale but if
you look at what's going on right now in
other parts of the AI community we have
actually made real progress in machines
that write code or machines that do
probabilistic inference in code so in
the short term I believe that it's not
just the thing we're gonna get to that
we're gonna have to earn and somehow is
going to emerge out of network right now
it is also on our route to building
explainable AI and to understanding the
explainable parts as well as the
unexplainable parts the more intuitive
parts of human time I wanted right maybe
we should hear your echo directly to max
problem that about how optimistic we are
for brain sparta AGI let me give you a
very simple example but very intuitive I
think I think you talked about how
optimistic are you about explainable
we're all really optimistic
I'm actually not not for the long term
because the examples like this I have a
two and half year old girl there's one
day when she said dad I'm gonna be a bad
girl I said what I'm gonna Punk you
I said no don't don't touch dad out I'll
get hurt so she sought a little bit and
then so this is why I want to bring
actually I am bringing my mind cognitive
empathy to my robot so that it can get
some autistic behavior so so I am really
worrying that once we have a model as a
brain spider cognate avenging that that
have general intelligence for as a human
brain in a developmental a way but one
day even with autistic behavior
she said II I'm not gonna do this
because I don't want to follow you
because people people do that starting
from year one they can they can they can
have their pre thinking saying I'm not
trying that oh so this is why III feel
we still meet some dangerous point
another example is humor
humans good of them from the broader
sense but talking about bias who have
bias human so when when simulating or or
building you see there's clouds there's
risks or there's limitations human
cognitive brain inspired so we should
study the neuroscience and cognitive
science of bias and how to avoid that
kind of a building block in how to
understand you well some of them might
be inevitable I mean and and something
in any kind of inductive system and
others might be avoidable and yet we
have to study them to understand that
yeah let's get more see first off great
great panel and and I like the the
discussion may part about the analogies
and learning from humans and human minds
and so forth and and all the advantages
of that I want to just give some counter
arguments in to have you react to them a
little bit and let me just first off
stipulate I understand that there does
exist this proof and there's some
benefit from the biology and it is arena
stead it makes it easier to communicate
and yahshua mentioned briefly that maybe
maybe it's less risky because it's less
likely to be bizarre but from an
economists perspective there are some
advantages to being very different and
first off I mean one of the reasons that
bulldozers or jets are so valuable is
that they have a very different ways of
generating power and so forth and that's
why they're much more valuable than then
then a biological version of those
machines and secondly maybe a little bit
more subtly if a machine is a closed
substitute for a human then it makes
human labor much less valuable
whereas if it's very different
complementary that has superhuman
capabilities and other dimensions but
not on some of the things that humans
could do it's more likely to lead to an
equitable distribution of income and
wealth it's more likely to have humans
still be part of the valuable ecosystem
so it'd be interesting to have you sort
of reacted for those those benefits and
cost of having systems that are very
different from us to versus systems that
closely mimic us and gradually just are
close substitutes so I can mention it
may be the analogy of birds and planes
so it's the same underlying principles
of aerodynamics
right but of course planes look very
different and you might argue that
planes are better in some way but
actually they're just optimized along a
different axis so birds have lots of
constraints of energy consumption that
are not as present for for planes and we
don't have planes that are as efficient
at flying in terms of energy consumption
so I think what's gonna happen is we'll
discover these principles will make
progress and then these will be of
course inspired by animal and human
intelligence but then we'll want to
apply that intelligence to other things
with different constraints than human
bodies have and then I think it will
happen as you say that will move away
towards maybe even before we get AGI
right it's not it doesn't have to wait
for AGI in directions that that are
going to be different that look
different in nature because we push the
in sort of application where the
constraints are different I don't think
I don't think that's what most of the
say deep learning community is doing
we're not trying to mimic humans so
there's a big difference between
mimicking humans as in the details of
what we do of our failures of our
neurons and getting inspiration and
trying to get to keep that inspiration
at an abstract level that corresponds to
where we think are the right principles
and maybe Josh and I could differ and
that's okay we need to explore all the
roots so that's that's not mimicking
right I think that complete the notion
of mimicking doesn't work can I just
come in but with an opinion and
moderation so I agree with points you're
saying both the value and the risk a
value of alternative paths and the risk
of a certain path is mimicking humans I
think another another way to put the
point is that I don't think any of us
are interested in bid or working on
building digital humans but when we say
weird we might there might be some
capacities of humans that we want to
achieve human-level
goals with human-like means
and it's for example common
sensibilities right if we or certain
kinds of social cognition that again
allow us to talk from each other if we
could have whatever kind of AGI we have
if we want to be able to interface with
it if we want to be able to talk to it
and Trust but but but that doesn't you
know that doesn't mean it has to be a
full person I think there is an
interesting point if you're actually
gonna build agents your point about
having a self essential but we don't
necessarily want to build you know AG
eyes that have a full human-like felt we
might want to have all human like common
sense but not a full stuff we might not
want them to have some of the same you
know fundamental motivations and will
that we have we and understanding and in
from an engineering point of view how
that actually works in humans allows us
he's a part of the components and
engineer the ones we want and not the
ones that we don't want to leave that's
one vision so I think that's important
again clear I mean to go back to the
popular bird Erol airplane analogy we
should build airplanes not but some of
the things we're building are also kind
of more like robot earth right like age
you know in some sense agents that like
might lead the flock in the right
direction if there's birds that are
flying or if there's robots that are
flying around with the other birds they
shouldn't be like airplanes because
either the airplanes where the birds are
both will die we've seen that happen too
we're liking this in the self-driving
car analogy we could just get any me
right now if we just if we pass the
global laws that banned all human
drivers and pedestrians and bicyclists
for the streets and said everybody has
to be in a self-driving car it'd be a
lot easier and safer in certain spans to
build autonomous vehicles but because
the route that we seem to be on is one
in which autonomous vehicles have to
live in a human world where there are
humans driving and walking and bicycling
and so on and leaving stuff on the road
and whatever then there's going to have
to be and I think most people that right
now agree that this is a key bottleneck
in autonomous vehicles they're gonna
have to have some understanding of human
intentions so that they can be safe and
valuable in interacting the world and so
that's the kind of thing that we're at
least I'm talking about that can be both
economically valuable landscape we need
to differentiate yeah let's let's take
let's look at
I've noticed that in quite a few of the
conferences I've gone through lately the
need to bridge the symbolic and the
neural based machine learning approaches
is is getting emphasized this as one of
the tasks and also that there's this
recognition that at least in the
learning of children that it's much more
embodied than much more robustly
embodied but in us humans evolution
forged us as creatures in which these
capacities are deeply integrated
integrated in ourselves and in our
adaptive relationship to our environment
and so my question is do we really are
we really just looking at these as
individual capabilities and trying to
forge them or do we really do all of
these tasks but don't integrate them
very well in an embodied relationship
their environment so my view is that
there needs to be integration and we see
that humans right we we see how our
language fades how the world right but
at the same time we can't as I mentioned
before which one is really learn
language at the same time as we learned
the kind of basic concepts so it's like
it seems to be this bootstrapping where
we first need to have some basic
understanding the world from which we
learn the symbols which language and
then language is what gives humans
disability to turbocharge our
intelligence because now we can move to
the symbolic domain and manipulate which
we create new structures just in the
symbolic space mainly using programs and
that so that that programs can only
exist in the space but that's the best
way humans kind of start overtaking I I
would say yes very much to what the
things you're looking for I would that's
the thesis that I'm talking about I
already have it very interesting
promising first
towards what you called neural symbolic
integration right it was art exists and
they're very important for embodiment so
you know I recently I found myself
working with a number of robotics groups
and when I work with robotics what that
mutually means is you know they do all
the hard work the engineering but I'm
very proud that a few robotics elaborate
we've published in the last year's a
couple of robotics papers two of them
have won best paper prizes that leading
robotics conferences you know I don't
claim any credit for that but what
they've done is they've actually in
different ways some of them like you
know are using actually sort of you know
basically combinations of symbolic and
the each of them used combinations of
symbolic and differentiable approaches
one of them has no neural networks at
all but as differentiable dynamics like
differential physics dynamics but then a
symbolic planner that basically says
which dynamic modes should you use that
was work at the market you stated in
another one which was worked at on araga
giant a mighty student did working with
Leslie Kelvin and others right was one
where you took an analytic symbolic
thank physics dynamics model and then
added on a neural network to learn the
residuals you can write down you know we
have really good physics engines that we
can write down right now which capture
your very powerful in capturing a range
of different kinds of physical
interactions but they're not perfect and
and there's many things that we don't
know how to capture their way so the
neural net learns the residuals and that
that is a very powerful combination also
that already exists and it's being used
in real robots so another example of
paper that will soon come out in science
robotics by nima is an example that they
plays Jenga that uses probability and
symbols to play Jenga rights but those
are real things that are happening right
now it's incredibly exciting
[Music]