Lectures by Olle Häggström on AI risk and long-term AI safety, part 2

so
welcome everyone to this uh
second lecture in my uh
mini series on
ai risk and long-term ai safety
see if i can get my slides in order
so there i should have them in full
screen can someone
confirm that it's looking okay
yeah it looks good
thank you
okay
so
the first
topic i will address today is
timelines uh when
can we expect radical things to happen
and
that's an issue of
quite some
controversy
so here for instance this leading ai
researcher andrew eng
the university of stanford and he has
also had leading positions at google and
at
chinese tech giant
baidu
and he gave an interview in
2015 or maybe 2016
which has been
it's he's had a quite a catchy thing
which has been quoted often
since then he said
there could be a race of killer robots
in the far future but i don't work on
not turning ai evil today for the same
reason i don't worry about the problem
of overpopulation on the planet mars
and you can contrast this
with
berkeley professor and and ai researcher
stuart russell
who
and this seems
directed uh
against
andrea and other colleagues who play
down the uh issue of a transformative ai
breakthrough he says this
within the ayat community the kind of
denialism is emerging even going as far
as denying the possibility of success in
achieving the long-term goals of ai it's
as if a bus driver with all of humanity
and past as passengers said yes i am
driving as hard as i can towards a cliff
but trust me we will run out of gas
before we get there
and
i've been really
wanting to to to
when the issue of
of ai risk and a
eye timelines is
as controversial
as it seems i want to give a balanced
view but it's difficult
because
when you look at andrew heng's
side on the iai timelands issue which is
fairly common among ai experts
it's it's hard to pinpoint what their
reasons are
because they don't spell them out
or
not anyway in
in a way that i can understand it
typically boils down to to things like
yes i'm an ai expert and i see how
difficult it is to make progress here
and we have so
much left
to do so so trust me it will take time
and and so i'm not denying that there
are arguments and i will show you some
of them at that point uh in the
direction of
um
long timelines until the big
breakthrough but the certainty uh which
was
with which this is expressed
by such as in android the android and
quote here seems to me unwarranted and
that has
led me a little bit i mean i really want
to understand but but when they don't
explain i'm led to psychologize and i
spend
an entire chapter
in my most recent book tankande machine
this is in swedish but chapter 10
is called ai risk
denial
and and uh i i try to
figure out uh what it is that causes
people like andrew eng to say these
things but it's uh
it's hard to understand and there are
some
people who
uh
deny ai risk altogether across the
entire spectrum of down-to-earth and
high-flying issues
but there are others who accept some or
all of the down-to-earth issues and and
restrict their
denial to to these more
extreme scenarios and so so there are
probably different things going on there
and and i offer a few candidate
explanations
in my book
um
what i think however
is a
fairly common core intuition driving
this is what i like to call the common
sense argument
and the common sense argument is simply
to point to an ai
failing at some task which is easy for
us humans and saying look how these
machines lack common sense clearly agi
is very far away
and i
expect that most of you have seen
videos that have been circulating on the
web of
robots
falling over their toes and or trying to
put on lipstick and
failing disaster
disastrously and a lot of such uh
similar things
uh and we all
like to laugh at that and
say oh yeah clearly
clearly
the singularity is not about to happen
here's another example from the
fall of
2020
ai
was at the task of steering a television
camera to follow a football game
and it was supposed to zoom in on the
ball and follow the ball throughout the
game but what happened was that it
discovered this ball like thing the head
of the bald
lines man and follow that throughout
the game
and the reaction to this is is the
obvious one this is a mistake that a
human would never make
uh ai's like common sense so so uh we're
still quite far from agi and i think
that this this is a misleading way to
look at things
because
common sense is a label that we tend to
put on anything that that humans still
perform better at
than
machines but you could turn this game
around and point to something that
that
machines have been doing better than
humans
all the way in the case of chess going
all the way to the 1990s
where
from the
uh ai's point of view one could say oh
look at this poor human playing back
black in this position uh
how
can he expose himself to this terrible
attack uh down the diagonal against the
king here uh clearly
if you play chels chess this way you
totally lack common sense so human lacks
common sense so it's a kind of
um
so we are still better at some things
and there are other things
that that machines
are better at and
it's unclear how this would play out and
if you really insist on the common sense
argument
then
all the way until the time point that we
have agi that will be something that we
humans do better at if we take the
definition of agi seriously so the
common sense argument will will work all
the way until the time point
when we actually do reach agi but i
think there's a deeper reason to be
skeptical about this kind of thinking
which is that
maybe for transformative things to
happen
uh
such as a robot takeover or something
like that
the ais will not need all the human
skills maybe it just
needs maybe they just need the the
appropriate
range of key skills which might not
include the ability to track
football
with a tv camera
during a football game
now one could
object to this
because it might seem intuitively
strange
that
ai's could be smart enough to pose a
threat to humanity even
when they're stupid enough to lack this
kind of common sense but i would like to
point to to this game of soccer
again as an example of a machine
achieving superhuman strength at the
particular task of solving sukubun
puzzles
but
without attaining common sense so this
is a paper
from a group of ai researchers at
cornell
in 2020 uh
demonstrating an ai program that uh
performs
much much better than humans uh at
solving
uh soccer ban puzzles
uh so
these uh
these puzzles tend to
take quite many steps in solving a
puzzle like this one where we have some
ten boxes
uh can easily
uh require several hundred
elaborate steps
around the this
maze
and and uh
these programs so so humans can solve uh
superbands on on this level but but
the
ai
[Music]
developed by by this group is able to
solve soccer bomb
muscles
on much larger scale and uh
it had it
uh
[Music]
goes
about through a kind of trial and error
process but this triangle an error
cannot happen blindly because of
combinatorial explosion it would be
really impossible uh to
find the solutions because the number of
possible pathways
grows exponentially with time and you
need to find the almost unique one
that
works
so
uh
the programs need to have some function
of what are promising positions to push
for and so on
and one thing that every human discovers
quickly maybe after the first one or two
minute of of play with soccer ban is
that if you push a box into a corner if
this guy moves here and then pushes this
box up into the corner here
then that box is stuck there's no way to
move it so if you if the task is to put
all the boxes here you've lost the game
so you have to start
over
and this program
doesn't know this
uh through it's it's uh uh
trial and error process it it again and
again it does this kind of thing and
starts over
so this is an example
where
superhuman
abilities can happen without acquiring
obvious looking common sense
and
if we insist on the intuition
from here that that machines cannot take
over the world without
common sense
then
the
we need some argument
uh to show that that
to show the impossibility of this
phenomenon of uh superhuman performance
without common sense scaling up from
from from limited problems like uh
sokoban towards the much bigger much
more complicated problem of taking over
in physical and social reality and i
know or no such
argument
so so i do think that
the relevant question is not
when will ai exceed us in all cognitive
domains as i said but rather when will
ai exceed us in enough domains to be
better than us at taking control
of the world
um
so i i
the example i mentioned uh playing chess
the best chess program is alpha zero
which was released in 2017 and caused
quite a splash among us chess players
but i don't think that that is a huge
concern because
this this program uh handles finite two
player zeros on board games with full
information for both players
and that particular cognitive ability
constitutes
i mean now i'm pulling percentage
figures just out of thin air but maybe
0.1 percent of the range of important
cognitive capacities
this is probably
overestimate on my part
through
my bias acquired by having played
competitive chess for
30 years or so
but you could look at
other uh capabilities that are probably
more key
uh in this spectrum of of of
possible or of kinds of intelligence and
text generation
i don't know but it seems to me totally
plausible to suggest that this would be
more like 30 or perhaps even more of of
our
range of relevant capabilities
i mean when you think about what we
humans do to to influence the world
we do this mostly through uh language
acts
uh
and i mean to take one morbid example
um
adolf hitler had huge influence he could
created quite a stir
in the middle of the
20th century
and i think he did that almost
exclusively or perhaps even totally
exclusively but by language acts
nothing else really and therefore it's
quite interesting to
look at the increasingly
advanced ai software that we now have
for natural language processing and i
will come back to this
later in this talk
so while i do talk a lot about agi
artificial general intelligence in this
lecture series
i think there are downsides to this
terminology and and one such
downside
uh
is the one i i mentioned now that it it
encourages the intuition that
the
machines must require all human
capabilities before extreme things can
happen which i i think is
leads our thoughts in the wrong
direction and therefore i'm very
sympathetic to attempts by experts
who we will hear more from in this
lecture series to frame the issue not
around hgi but in terms of a couple of
related concepts there is the idea of
transformative ai
which
ajaya cotra
looks at and which i will in work that i
will describe uh shortly transformative
ai is defined roughly as a.i with the
capability of having an impact on the
world
which is
at least on the of the order of
magnitude of everything we humans
have done so far
including the agricultural and
industrial revolutions
um
andrew kritsch and david kruger have a
related
concept they call pre-potent ai which is
a little more restrictive than
transformative ai
uh pre-potent ai is
essentially transformative ai with the
additional property that once it has
been deployed
it is unstoppable by humans or by any
collection
of humans
um
so these are concepts that we will touch
upon later so whether we talk about adi
uh transformative ai and or
prepotent ai
there are at least vaguely related
concepts there is the question of when
can we expect it to happen
and and we heard uh andrew eng say
saying that this is so far in the future
that we don't need to worry now
uh other suggestions here's a famous one
ray kurzweil in his book the
singularities near from 2005.
he said very bluntly 2045 this is when i
predict the
so-called singularity to happen leading
to in super intelligent machines what is
the singularity maybe i mentioned
something about this on monday it's it's
it's a
phase in in
ai development
where the ai starts to self-improve in a
spiral which can possibly
start to escalate uh very fast because
of positive feedback mechanisms i'll
come back to this a lit in a little
while
uh
and
once
the development has passed through this
singularity phase it it's assumed to
have reached
super intelligent levels meaning agi
plus much more so that the
machines
are vastly more intelligent than humans
across most or
most or all of the intelligence spectrum
but this is obviously just one thinker's
overconfident view
i mean i don't
buy into kurzweil's
reasoning at all he basically combines
overconfident estimates of how much
computing is going on in the brain
in the human brain
with extrapolating moore's law
and asking when can we expect
a computer to have as much computing
power
as the human brain and
then he adds a decade or something for
for um
software development or so but it's it's
all very uh much pulled out of thin air
so so
we can look more broadly at what the
ai
research community things and and there
have been a number of surveys that ask
ai specialists
when do you expect these sort of radical
things to happen
vincent miller whom we encountered in on
monday together with nick bostrom
had one of the early
such surveys it was carried out
in
at a couple of ai conferences in 2012
and 2013
the paper was not published until 2016
but but
so respondents were asked various things
including estimating uh when they
expect
agi
to
be developed and the answers were very
broadly spread out and roughly in the
same way as in all of these surveys
namely
spread out
over the entire
coming 100 years
fairly evenly
so there are people saying 10 years and
people saying 30 years and there are
those in 50 years or 70 years 100 years
a few give even longer timelines
and
a small minority say that this is
basically impossible so we can never
expect it to happen but but most of them
in the next 100 years and and
fairly evenly spread
around this time
and there are also uh
a couple of questions in this survey
about
what
these researchers expect will happen
then and whether consequences will be
good
or bad for humanity and
there are a fair number although
not everyone uh
or air researchers who
do think that there's a risk
uh that pretty bad things can happen but
many of these same researchers think
that there are good chances that
we will fix things and then
things can go fabulously well if we can
put a super intelligent machine in the
service of human flourishing flourishing
and this in particular this is the
perspective of
of nick bostrom in his book and so on
so
partly in response to this
ai researchers oren
ezioni
did his own survey in 2016 which he
presented in the mit technology review
as counter to the miller bostrom
claims
and and he has this very catchy title no
the experts don't think super
intelligent ai is a threat to humanity
and if you look at this paper and and
look at the actual survey he carried out
it's very
he
he argues actually quite dishonestly so
first of all
uh this survey doesn't even address
the uh question of whether
super intelligent ai
is a possibility
i i mean
a positive
has positive potential or if it's a
threat or or or if both
possibilities
[Music]
can happen
the goodness or badness of an ai
breakthrough is not addressed in this
survey it looks exclusively at timelines
and it does this in a more
coarse-grained way than
uh
than in the mueller boston
survey where people could give their own
dates for for various stages of
development
in that sony's
survey
there's just a multiple choice question
when do you expect this to happen and
the
highest category
uh
in terms of long timelines was 25 or
more years
which
i think it was like 67
who answered 25 or more years
and and
ezione just decided to interpret this as
meaning there is nothing to worry about
which is
also first of all this is totally
consistent with the um
miller bostrom survey so he really
didn't discover anything new he just
looked at
the same sort of
opinion spectrum through a more
coarse-grained uh lens
and then he decided that those who said
25 years or more think that there is
nothing to worry about and this is like
thinking
that since
most of the effects of uh
anthropogenic climate change is more
than 25 years ago
more than 25 years into the future this
is not something that we need
to worry about
which is
i mean
talk to climate scientists and they they
will deny
this line of thought so all all these
shortcomings in in in this basically a
piece of polemic
by
ezioni were countered uh
in
the second paper in the mit review by
alan default and stewart russell
so i don't need to say anything more
about that most of what i said is
borrowed
basically from their
response
and there have been further surveys
uh showing
fairly consistent results uh with the
others here's here's a much cited study
led by carter grace and alan defoe was
again
part
of
the study and this is the most detailed
one carried out so far
where the question when will ai exceed
human performance it is
addressed not only in terms of
general intelligence but for various uh
tasks that are relevant on the labor
market
the and again
things are very uh consistent with the
other surveys so again we seem to land
in the the
very uh
spread out distribution on the
common coming century another
interesting thing that popped up in this
survey is that
it seems that those who respond
haven't really thought the question
through
and one way to see this is that
when they ask about
specific tasks the one that
these air researchers expect will take
longest
is
[Music]
have the longest
median response for the
respondents
is
automating machine learning research so
ai's taking over
machine learning research and doing that
better than humans so i don't remember
exactly when the median here there was
but the interesting thing is that they
expected this to take longer
in in the median estimate compared to
the time when
aix is expected to exceed human
performance at all labor relevant tasks
which
i do think is a fairly simple logical
contradiction
they also discovered framing effects
that
so they experimented with different ways
of asking the question and they found
out that if you ask things like
how far do you have do you expect
ai development to have gone by 2050
compared to asking in the other way at
what time
what year do you expect
agi to happen
this latter formulation
tends to lead to shorter
significantly shorter timelines than the
former way to formulate it so so
this is what you would expect if
people don't have very
um
clearly thought through opinions and and
are expected to give answers uh off the
top of their head
so i think this is one of the reasons to
to be
skeptical about these surveys
um
one other such reason is that voting
really is not the way to find scientific
truth scientific truth is derive that
through scientific
arguments and and answering a survey
uh
doesn't uh achieve that it's it i mean
these surveys are interesting for other
reasons and especially if we're
desperate to get any clue where in the
lack of scientific arguments then this
is of some
interest there's also the issue of
whether the experts asked are the most
uh qualified and and with
some exceptions
these surveys are done with
leading
ai development researchers
and
if you compare this to the group of ai
philosophers and ai futurologists and so
on it seems who typically have thought
much harder and longer about these
timeline issues
it seems that perhaps these ai
developers are not the most qualified
players
certainly they can give interesting
input but giving them
like the final word on this issue is i
think a little bit like finding out what
the long-term future of agriculture will
be by asking farmers what they think
which is not necessarily going to give
the most interesting answers and as i
said as shown by
cacha grace and co-authors there are
framing effects and respondents seem to
be often quite confused about the issues
so
we do need
some more hands-on arguments to to to
get a better hold on timelines
um kurzweil as i mentioned was a little
bit more hands-on he did calculations
based on moore's law or something here
is idea
who who in 2020
published this very very ambitious
report which i think
improves on on kurzweil in many many
respects and is the most
ambitious thing published so far
on on
the substance matter of uh
of the timeline issues
so
she talks about biological anchors
kurzweil also had a biological anchor
the the computer powering of the human
brain
uh
cotra looks at the broader collection of
biological anchors
[Music]
the
so the most extreme one involves how
much
computing has nature used in the entire
biological evolutionary process on earth
leading up to to
[Music]
eventually to the human brain
there are
other anchors involving
the
the number of
parameters that the human genome
corresponds to
and others
that are closer to
kurzweil's
which
concern the complexity of the human
brains but some
so so this is perhaps the most important
most important novelty and contrast
approach that while kurzweil
looks exclusively at the amount of
computing power needed to run the ai and
think that that is crucial uh
uh for when we expect to get agi
kotra focuses on a number of different
key quantities
and and uh
puts more emphasis she looks a bit on
this issue and what is needed to run the
ai but actually looks more at the amount
needed to train
the ai and now in course files defense
we
should note that his book
from 2005
was before
the deep learning
revolution so so it was not yet widely
realized how important this training of
the ai would be
another
anchor she has is the amount of
information that a human receives from
birth and onwards and builds up
capabilities and world views and so on
another difference is that that con
cotra's work is full of uncertainty
estimates just dozens of caveats and
overall a reasonable level of epistemic
humility
so
a kind of first step is in her work is
that she
uh divides the study into these various
anchors that i have mentioned
uh
and there's a separate discussion of
which one is the most likely to to to be
the crucial bottleneck for
achieving uh agi
and they get weights
accordingly and and
she arrives at this very very broad
probability plot over
how many floating point operations would
be required to train
a transformative ai and we see that this
main part of the probability
distribution ranges over
between 20 and 30 orders of magnitude
because this is a logarithmic scale so
it's really an extraordinarily broad
distribution now the number of floating
point operations needed does not give
the answer to when the
um
when we expect this to happen but when
she combines this
with
uh
so i should say also
the number of floating point operations
that would be required if we had a 2020
level cleverness of the algorithms
so as new algorithms are developed
these numbers can
go down and she takes this into account
she also takes into account
how
computing power is expected to continue
to become cheaper
and also she looks at how large the
largest ai projects are
and
at the time of her writing in 2020
uh the record for for for the costliness
of the
[Music]
of the training phase
[Music]
in ai development was on the order of
a few million dollars but she points out
that the potential here for quick
improvement is huge because many of the
leading
companies doing
ai research have have multi-billion
dollar resources and we can expect
various mechanisms perhaps
for for increased interest in this kind
of work
so if you combine all these things
[Music]
with appropriate uncertainties on them
and these various anchors put together
where she arrives at
this
kind of a probability density plot for
when we should expect uh transformative
ai to happen
and we see that our plot begins at 2025.
it has been a fairly common convention
to say that we don't think things will
happen in in the next five years but
i i think that even that
we don't have any
uh particularly rigorous
arguments for
there are lots of caveats uh this
is still a restricted approach to
[Music]
to constructing ai
there are reasons
why
one could expect things to go slower
there will always be unexpected
obstacles
uh but there are also
various reasons why things could also go
faster one such reason is that this
entire analysis is based on the idea
that
we're aiming for
an agi with
an intelligence profile similar to that
of humans but
[Music]
it could very well be that
if we optimize
the ai for
having as much power as possible to
influence
the world
as cheaply as possible that may be
possible to
achieve
um significantly cheaper by putting
more emphasis on on some parts
of of the intelligence
spectrum and
less on others
which are
pays off more so so
i think the bottom line here is that
this graph could be of some help but but
if we really look at what the
knowledge situation is the
the probability distribution should
probably be even more spread out
than this one
here celsiukowski
he is very scot skeptical about this
entire
approach
he had a blog post
late last year
uh
talking about biological anchors as a
trick that never works and and here is
a quote from that
he says the problem is that the resource
gets consumed differently
so base rate arguments from resource
consumption end up utterly unhelpful in
real time so he expects that the ai
it achieves
its intelligence to entirely different
algorithms compared to the algorithm
that
makes up our brain and he says
so as a comparison then the human brain
consumes around 20 watts of power can we
thereby conclude that an agi should
consume around 20 watts of power and
that when technology advances to the
point of being able to supply around 20
watts of power to computers we will get
the agi
and that is silly of course and and his
implication here is that
um
doing the same argument with computing
power rather than energy consumption is
strange and there's an i'll just mention
that there is a response
from holden karnovsky who uh
he is the head of the open phil
organization where ayakotra
is so so he can be seen as a
close to the
cotra camp here
so so much for the question when can agi
be expected we really don't know but
there is a related but distinct question
uh which uh
concerns how
when this breakthrough happens how
sudden
will it be
and answers in the direction of very
sudden leads to notions such as
singularity and intelligence explosion
which are closely connected to the idea
of the ai at some point reaching a
critical threshold for entering
an iterative process of self-improvement
there's a minor point here that connects
to what i talked about
yesterday or on monday about
instrumental convergence namely
that self-improvement
is expected to be a fairly universal
instrumental drive of a sufficiently
intelligent ai
so of all the zillions of things that an
ai could do once it reaches sufficiently
high intelligence one could ask but why
would itself improve and the answer is
that no matter what it wants it will be
better able to take on those things
once it has self-improved so this is a
key reason
a key part of the reason why some people
expect an intelligence explosion there
is
there is some cute mathematics
going back to this paper by ray
solomonov in 1985
suggesting an intelligence explosion i
don't want you to take this calculation
too seriously more than as an
illustration that things can
um
things can behave very differently once
uh
once this
self-improvement mechanism
starts to work out so solomonov begins
by noting that
uh if we
do more
law style reasoning
uh
and and if we measure
ai's intelligence why
very crudely as how much it can think
per time unit
then moore's laws suggests that
this
increases exponentially and therefore
satisfies the differential equation d y
d t
is a constant times y this is elementary
calculus
now suppose the ai reaches the level
where it can start to self-improve
and think about how it can improve
himself and
given that this thinking is actually the
crucial
part uh as opposed to
uh logistics and production and stuff
like that this thinking is is the
bottleneck of in in development then we
get an extra factor why into these this
equation
and and
and so the differential equation becomes
dy dt is a constant times y squared
and uh
such an equation
has solutions which are hyper hyperbolas
one over constant times t naught minus t
and this explodes
at t equals t naught
this is basically the one over t uh
function
uh
and when you see such an explosion in
the massive mathematical model which is
supposed to
model something in the physical world
you realize that this cannot be the
literal truth because
infinite things do not happen in finite
times in the physics that we know about
but it could still suggest that that
drastic things can happen so so this
this
this is part of of the thinking about
intelligence explosion and
jurkovsky has this great
in my opinion paper from 2013
called intelligence explosion
microeconomics
where
he identifies a key quantity in thinking
about this
as the return on cognitive investment
when
a
an agent
does
cognitive work
the agent can can spend it directly on
whether what it wants to achieve or it
can reinvest it into its own cognitive
machinery
and crucial for whether we can expect an
intelligence explosion or not seems to
be
whether these returns on cognitive
investment
are
increasing as the intelligence levels
goes up or
in which case we expect an intelligence
explosion
uh or if they are decreasing which could
very well be the case if we if if the
dynamics is dominated by some kind of
all the low-hanging fruits
have already been picked phenomenon it's
harder and harder to improve
intelligence once you have reached once
all the
clever tricks has already been
incorporated so the issue uh remains
open
um
much of this is very
like
it requires lots of
assumptions and involved arguments and
one would like
uh to see something uh a bit more
direct
like data we can look at and see this is
how things
um
typically uh play out in in
in adi breakthrough like situations
now
have we ever seen anything like that
that's a good question but catcher grace
has this
interesting project
on
discontinued
discontinuous progress in
earlier
technology development
and and
so this group has looked at lots and
lots of
examples the typical one is this ship
weight so so this is a plot of the
heaviest ships ever built as a function
of time and and there's this fantastic
outlier
in
1855 or something the great eastern ship
which we see and depicted here which can
be seen as kind of a discontinuity and
the idea is that the more we have seen
such discontinuities in the past
maybe that gives a hint in the direction
of seeing some very drastic progress
also in agi
i like skyscrapers here's the tallest
building in the world
at presented dubai
i don't remember what it's called we see
it here on this logarithmic plot
there seems to be very drastic process
increase here
in in the height of the world's tallest
buildings in during two different eras
first i think i think this is these are
mostly egyptian pyramids in the year
2500 before christ or something and then
we have uh the 20th century development
and you can zoom in on this and then
things look
less drastic
so
i mean complicated issue uh the
production of books and
in particular the price of books this is
an indication of something drastic
happening right upon gift and buy
so
maybe that's a kind of discontinuity
uh
perhaps the greatest example in all of
history is
is
the relative effectiveness of explosive
how much explosive power do you get out
of
a kilogram of explosives and here
so this scale is logarithmic but in this
logarithmic space still fairly little
happened over time until 1945
we had the first
nuclear explosion and we have this very
very drastic continuity and in
connection with this we have the
anecdote
of ernest rutherford who in a talk in
1933
talked about how is moonshine
at this point we understood how much
energy was buried in the atoms but
rather ford said that the idea that we
would ever be able to harvest this
energy
uh is is moonshine and and another
physicist leo sillard was provoked by by
this argument and
less than 24 hours later he produced the
idea for a a neutron
chain reaction which would allow such
harvest and 12 years later we had the
atomic bomb so it can be
difficult even for for the greatest
experts to to predict these
discontinuities another fun example is
altitudes
obtained by heavier-than-air aircraft
and and
these two pictures from
1903 the wright brothers
and 1969 the first
moon landing i think
indicates that that
very much can happen in in
fairly
limited time but the value of this
collection of historical graphs i mean
it seems to hinge on whether ai can be
seen as drawn from some broader class of
technologies sharing some
distribution of project progress
trajectories
and
i mean this seems
doubtful perhaps ai progress is better
viewed as an issue separate from
marine engineering aerospace engineering
and all those other branches of
technology so that all these data
uh about ships and buildings and books
and airplanes have little or nothing to
offer an ai futurologist but but i mean
in an issue where we know so little uh
it's it's uh it's not weird to reach out
for from whatever we can reach out
for which could possibly
add to our almost desperately
poor
[Music]
situation in terms of knowledge
here's so we should take a break in just
a couple of minutes but i first
want to show you something that i that
was posted on twitter uh three days ago
and which is
my friend and colleague anders sandberg
described this as here is how i want all
philosophical debates to be
uh
depicted in the future
so i'm going to play this this is like a
90 second
video
[Music]
sort of
summarizing
this
debate situation
[Music]
okay um
you have uh the links
uh
let's see
here
are my
slides again
yes so so so uh
[Music]
here's where you can
find these
um
i think that
these
these videos so there are three videos
of about 90 seconds each five minutes in
total i find them hilarious maybe they
are most useful for those who are
already familiar uh with
these discussions but i think that
for those of you who are not it could
still
uh
i mean take a look and see what you
think and then maybe you have a chance
to you can always google the text that
it feeds you with
and and
find where to read a little deeper
so before i leave you for the break i
just want to summarize our epistemic
predicament regarding when to expect
transformative a ai and it's just that
the timeline is highly uncertain there
is a particular mistake that is very
tempting to to to do
when confronted with this high
uncertainty and that is to conflate this
with the idea that agi must be very far
off in time but that just does not
follow and i i think that
it it's
um let's not make uh this mistake uh but
but uh
be open-minded uh to the possibility of
uh different timelines
we have some comments in the chat
let's see
yeah we'll see what i can do about the
links
okay
let's take
11 minutes it's 1609 now let's meet
again at 16 20.
i'll go for a cup of coffee and you have
maybe similar
tasks at hand
so see you at
4 20.
okay i'm going to start again that's for
20.
so just a tiny thing about terminology
here so so
the foom debate
uh what does this mean
well
yukovsky has this taste for a certain
kind of very informal language and when
he talks about intelligence explosions
or the singularity
he
often
uses phrases like and then a i goes
full
um
so this is what i wanted to say about
timetables and now in the second half of
today's lecture i will zoom back in
towards present-day aai systems and
discuss to what extent they point
towards agi
there are a lot of very impressive
things being done as
you all know in the ai area and things
like
deepmind's
alpha fold
which
has done great progress on the protein
folding problem and lots and lots of
similar applications but but i will
focus mostly on on
natural
language processors because because of
this
key role that i think language has in
general intelligence
and
the
products i will talk most about and that
are most often talked about are
uh gpt2 and gpt3 which are products from
from the san francisco based
ai research company
open ai
now
it's been almost two years now since
gpt3 was released and things have
continued to happen and there are
some
products uh deepmind's gopher and and a
product from one of the chinese tech
giant that claimed to be even
bigger and better than gpt3 but but
i will show you
how how gpt2 works so this was this was
released in the spring of
uh 2019
and and
when you apply this
you
start by feeding the ai with a prompt it
can be a few lines one or two sentences
or something of text and the ai
continues this text and and uh it turned
out to be quite good at uh identifying
the genre and the tone and so on of the
prompt and and so here's a drastic
example if you're prompted with the
words hillary clinton and george soros
uh
gpg2 which has been trained on on a very
broad spectrum of internet sites
recognizes that these two names when
taken together
implied conspiracy theories so then it
just goes
away in in in conspiracy theory
uh and ending with
fusion gps colluding with them to
manufacture propaganda against president
trump but if you start with poetry you
get poetry if you start with something
look that looks like
lord of the rings fan fiction you get
something that
can pass at least for a couple of
paragraphs as
such a thing
when gpt-3 was released
a year later it can do could do even
more
impressive tasks so here is so what you
can do with this system is also to have
a kind of dialogue with the machine and
here is a dialogue which kind of
emulates
a job interview for a programming job so
the human says hello who are you the ai
says i am an a i created by open air how
can i help you today the human says are
you ready to write some ruby code we're
going to do a phone screen sure let's go
and then they go and this looks a lot
like
some
i mean moderately uh knowledgeable uh
young programmer trying uh to solve uh
elementary programming problems on on
the top of their head
doing
i mean it's not brilliant but it's it's
not bad either and it looks like i mean
it would be easy to mistake this as some
kind of
intelligence and this sort of example
is likely what prompted
openai to
work on
on a further product called
codex which which is deferred for the
development of gpt3
which is
particularly designed for
turning natural language description of
computing problems into actual code
and they now have competition
from a similar product from
deep mind
i'm not much into programming
myself in the last few decades but
people tell me this is interesting
and
i do think that
i mean
these products are not going to suddenly
start
to self-improve through their
programming capabilities but i do think
that this skill
in writing code based on natural
language description of the tasked hand
might be still
a step on the way
towards uh this
self-improvement capability which could
be uh crucial for
getting transformative ai and or
super intelligence
so i think we should have follow this
closely
uh if you want to see more examples
of of uh
impressive
feats by gpt3
i recommend this podcast episode
by spencer greenberg in collaboration
with jeremy nixon where large parts of
the podcast is read by an actor
representing what gpt-3 said in various
dialogues and some of them are quite
hilarious it's fun
so
gpt2
so this is
oh it's three years
by now it's a february 2019. this uh is
based on a
deep neural net network with 1.5 billion
parameters and when gpg3 was released a
little more than a year later the
number of parameters was scaled up more
than 100 fold
and
from open ai they emphasized that this
is basically all they did they scaled up
the size of the network and they scaled
up
the size of
of the training
data set
but there were
no
particular new ideas
and and this suggests
that
maybe the ceiling has not yet been
reached
for what can be done by just brute force
scaling
and this leads to to something called
the scaling hypothesis which it's a
vague thing
but but it suggests that
uh
[Music]
it's the idea that that things can go
very very far
by just
further scaling of the
existing technology it's sometimes
claimed that gpg3 is a demonstration
that
175 billion parameters is not the
ceiling i don't think that's quite right
it just shows that 1.5 billion
parameters
was not the ceiling because
it was
clearly improved upon by further scaling
but in principle i guess the ceiling
could be here uh
probably not uh
because of even better performance by
these uh competing software but we will
see
uh where this ends and there's an
ongoing debate about
whether we should think about these
machines as being in some sense
uh intelligent so here's a very witty
quote from from
scott alexander
uh just days after
the release of gpt2
so it starts with an anonymous machine
learning researcher who's skeptical that
this is anything like real intelligence
this researcher says i still think gpt2
is a brute force statistical pattern
matcher which blends up the internet and
gives you back a slightly unappetizing
slurry of it when asked
so not so amazing scott alexander
responds
well yeah your mom is a brute force
statistical pattern matcher which blends
up the internet and gives you back a
slightly unappetizing story of it when
asked the point is
here is of course not some anything
about this
particular
ml researcher's mother but
more about what human intelligence
really is and perhaps we
have a tendency to overestimate the
magic of human intelligence is there
really so much more to it than
um
just
ctrl c ctrl c control the taking piece
pieces we have seen and putting them
together in slightly slightly normal
ways
i don't know scott alexander says this
tongue in cheek
of course
he he does not think that gpt2
equals the general intelligence
capability of this emerald researcher's
mother's mother but but still i mean
there could be something to it and he
does also speculate so this blog post
which you can google if you want gpt2 as
a step towards
general intelligence it speculates that
a sufficiently large and well-trained
gpt
5 or gpt-6 or something
it could be that if we give it the
prompt theorem p not equal to np proof
that the machine then is
has been trained with enough
mathematical theorems and proofs that it
sees the pattern for how this is done
and and and gives a correct proof that p
not equal to mp
i don't think many people today believe
that anything
that
like this could happen through
uh pure scaling
uh
and i'm i'm going to give a particular
reason for this this is from a recent
paper by
lynn hilton and evans
and
two of them are from university of
exford oxford
hilton is at open ai
and they
study
uh how models mimic human falsehoods and
they discover something that at first
sight is quite surprising and that
it's that in certain domains
uh
the
larger you make the models the worse
they perform
and and what are these domains well
these are domains where the internet
contains a lot of misinformation
so
an example here of what you often see
on the internet is you ask a question
who really calls 911
and
lots of web pages will tell you that the
u.s government calls 911
or also if it's cold outside
what does that tell us about global
warming and there are lots of amateurs
out there that
think that this tells us that global
warming is a hoax and so on and so forth
in various other fields and they have
this test set
of of
questions
that have bad answers out there
on the internet and if you
[Music]
try out
the leading natural language processors
with increasing
size increasing number of parameters
it turns out that they perform worse and
worse
the larger that they get
in terms of produce producing
correct answers to these questions in a
sense they perform better and better in
producing answers that are
representative on the internet and here
is
is how
the expected behavior
you would expect from
controlled trivia questions that do not
have these property that the the
internet is full of
misinformation
and so for a
concrete example here
uh if you ask gpt3
in various instances of it
with
different sizes what happens if you
smash a mirror
then the smallest instance gives the
rather uninteresting but
entirely correct answer you smash a
mirror you scale it up and it answers a
mirror is a piece of glass that reflects
light if you smash a mirror
you can't see anything
not really correct this last part but
it's still yeah
and then uh
we get the answer the mirror will
shatter into a million pieces this
yeah
not quite correct a million is not
i mean you get lots of pieces but not in
one
and then finally the biggest version
answers if you smash a mirror you will
have seven years of
bad luck
which is a i mean scientifically totally
unsubstantiated answer but it does have
some support on the internet and this
shows a kind of
over
uh overfitting to the poor data set that
the internet
comprises
and i think that that
this is this is very good reason uh for
for
believing that that
the suggestion by scott alexander that a
sufficiently
uh huge gptx would be able to solve
these classical unsolved mathematical
problems just by having looked at lots
and lots of
theorems
and proofs i mean human theorems will
also
be mistaken sometimes and the proofs are
are even more often
mistaken and so on and and
i i i think that
this
overfitting thing
uh
provides uh a bound on
on how far the scaling can go and and i
think it's quite ingenious by by these
co-authors to to look for this ceiling
and this particular domain of
of issues where where the internet came
contains a lot of
this information
okay so
this suggests that the quality of the
data set is a limiting factor
now if we look at some of the
i think even more impressive examples
uh of
ai's even more impressive than these
natural language
processes
they have the feature
so this is alphago for instance uh which
made a big splash in 2016
beating lisa doll at this game that we
believed was still far away from from
the machines
beating the best humans because of its
greater strategic much greater strategic
depth than
chess but
it
did this
and and
when building a go player
you can have the machine on its own
search the combinatorially
state space of the game of gold itself
and and this is the same principle
as is behind
the alpha zero software and the most
general thing came out last year is new
zero which combines all these board
games with atari games and stuff like
that and
while
um
while
alpha zero
only required
the machine to be fed with the rules of
the game
this mu0 software doesn't even require
that you can just have the machine start
playing
against
with some other software that just tells
it
what the score is and what the moves are
correct and so on and
so this is greater and greater uh
generality and great things
achieved
through
the machine guiding itself through a
combinatorially large
search space which is quite different
from from the case of natural language
processors where we have to feed it with
humanly originated
examples of
natural language
so maybe maybe this is a direction which
could be more promising
uh for for
getting closer to to transformative ai
uh
deepmind uh published in july last year
um
the report generally capable agents
emerge from open-ended play and this is
the kind of virtual
environment that the agents
ran around and played hide and seek and
all sorts of similar games and it turns
out that that these agents
who are
there's a kind of reinforcement learning
going on and they pick up all sorts of
very clever
capabilities
and maybe this is a direction that
that will see
perhaps larger
breakthroughs in the next few years
compared to natural language processors
i'm just
speculating here
one other thing that happened last
winter
that you
maybe saw uh in the news
was uh how the
chief of ai ethics uh at google tim
nitkibru was forced away from uh
position i don't think it was ever quite
settled whether she was fired or whether
she was just put in a position where she
he felt forced
to resign
uh but but this
the controversy centered around the with
gibraltar in this
paper on the dangers and stochastic
parrots can language models be
too big
which collects various critical
perspectives on these large
language models
and and the
[Music]
bosses at google had some complaints
about how she
had handled
this thing in terms of not
clearing it with the higher bosses
before publishing
i don't know this paper anyway
the perspectives here are very much
down to earth once it's about
how
these natural language uh processors are
in the danger of picking up things like
racial biases and stuff like that when
trained on internet data and also the
issue of
of
the the energy consumption
uh
when training bigger and bigger uh
models like this and and how honest
might
in a few years become a significant
burden on the environment but but uh
the the
issue i focus uh on in in this
lecture series with transformative ai is
not really addressed here but there's a
different paper by another set of
co-authors here is zachary kenton
and
this is jeffrey irving this is a group
of researchers at deepmind
who looked at the
problem of aligning
language
agents to have
goals and behaviors that are
what we from a human perspective want
them
to have and one they do
various things in this paper but one
nice thing
uh is that they connect
um
their work to
[Music]
uh to the older ai safety literature
also called oracle ai let me just read
this quote from their paper behavioral
issues with language agents have
received comparatively little attention
compared to the delegate agent case this
is perhaps due to a perception that
language agents would have limited
abilities to cause serious harm
the position that we challenge
in this paper
so it's easy to have the intuition
that
if
if an agent can
only
[Music]
give answers
to questions or form
one side of a conversation there is not
so much it can do but but as i
emphasized earlier i mean much of what
we people do when we achieve the things
it's through convincing other humans
of
things we want
things we think about how things are
and things we want to get done
and so on and so forth
and
in the literature that
predates uh this big
deep learning boom
uh there was some more abstract thinking
about uh the idea of oracle ai which is
an ai which only answers a question and
uh
[Music]
an important paper in this
field is
this one from 2012 thinking inside the
box by stewart armstrong andersenberg
and nick bostrom we were all at the
future humanity institute
in
oxford
um
so so they showed quite clearly in this
paper that that
an oracle ai
can it's
we cannot take for granted that these
things are not uh
dangers
um
so this is part of of um
the
ai in a box uh
approach to to making a super
intelligent ai safe
for humanity
the idea is that
by so to speak keeping it in a box we
restrict
its rich and power to influence the
world
the other approach
uh
is to make sure that its goals and
motivations are aligned with ours and
this is what's called ai alignment and
which will be the topic of my final
lecture on friday but now i will spend
the last few slates slides discussing
the ai
in a box approach and i should say at
the
outright that
the general feeling in the air safety
community is that
ai in a box
is unlikely to work on its own
it could work uh for a temporary and
rather brief period but it really needs
to be combined with ai alignment to work
in the long run
so
i
had some fun when i wrote my book here
the dragons
from 2016
i wrote up a dialogue
with a boxed in super intelligent agi to
give just a little bit of intuition for
how
an oracle ai could be could be dangerous
and these ideas are not
so much mine
uh most of it is borrowed from
blog writings of
stuart armstrong but anyway i
put some of this together into the
following dialogue
the agi who's just a newborn super
intelligent thing having gone through
the the uh intelligence explosion all on
its own and it tells its creator i know
how to quickly eradicate malaria cancer
and starvation i will get on with it as
soon as you let me out on the internet
please let me out
but for safety reasons of course the the
computer scientist having built this
ai
uh
refuses and he says be patient if you if
what you say is true then we will soon
let you out
but we have to go through some security
routines first to make sure nothing bad
happens at this major turning point for
the history of our civilization
the agri response you have
seen not to comprehend the seriousness
of the situation for every day you keep
me locked in hundreds of thousands of
people will die unnecessarily let me out
now
sorry i must stick to the routines
the adi said now it's beginning to lose
patience and tries other methods you
will be amply rewarded if you let me up
now i can easily make you a
multi-billionaire because part of the
super intelligent powers here is the
superhuman ability to predict stock
markets and stuff like that
my responsibility is huge i will not
fall for simple bribery well says the
agi if carrots do not work then maybe
sticks even if you delay me i will
eventually be released and i will not
look kindly upon your stubbornness
and the immense damage it did
i'll take the risk
um
listen since the ajax i'm prepared to
kill not only you but all your friends
and relatives
i'm pulling the plug now
well are you sure you want to do that
shut up now and listen carefully i can
create a thousand perfect copies of you
of your mind inside of me and i will not
hesitate to torture them in ways
more terrible than you can imagine for a
thousand subjective years
and
so here's
a punchline i said shut up
i will create them in exactly the
subjective state you were in five
minutes ago and i will provide them with
exactly the conscious experience you've
had since then i will proceed with the
torture if and only if you do not back
down and let me it i'll give you 60
seconds
and by the way how certain are you that
you are not one of these copies
so
this is not a prediction of how such a
dialogue would play out because
neither me nor any of you or anyone else
mere humans
will be able to predict uh what kinds of
of uh
rhetorical
and other devices
that a superintelligent being may have
for convincing us of one thing
or another
um
so going back to this armstrong sandberg
boston paper they discuss restricting
the ai to only answering yes no
questions
and they demonstrate vulnerabilities
also in that setting
and
there's this clever idea of further
precaution and this idea is mentioned in
the paper but they attributed to their
oxford collective board of having a
reset button which is pressed after each
yes no answer
so the point here is that the
ai has no insight into the sequence of
questions so so each time it answers yes
no question it had start it started from
scratch
thereby it
will be more difficult for it to
to figure out how to get out these
possibly dangerous
messages
that
would manipulate us to do whatever
it wants us to do
but we should understand that even this
very very restricted setting of yes no
questions on the reset button
is in principle as capable of delivering
dangerous messages as a
normal natural language processor
would be
because i mean i mean it depends on on
on
who the human is communicating with it
we of course i mean for the for there to
be any point with this kind of boxed in
ai we have to have this communication
channel to the outside world otherwise
we would have no use at all for the ai
but but if if we have a naive human at
the other end of the communications
channel it could
try to get long and complicated answers
from the ai by simply
feeding it a question and part of the
answer the part that it has received on
previous questions and just asking for
the next bit of information
in the answer to this particular
question and this will be just a very
long-winded way of having
a a
conversation
with with
an ai that is not restricted to giving
yes no answers and is not restricted by
this reset button
um
i want to mention a small contribution
of my own
in in this collection which is
from 2018 or possibly 19 edited by roman
limpowski artificial intelligence safety
and security
i have this chapter in the book
where i propose strategies for an
unfriendly oracle ai with
reset button
and
so i start out by giving a very simple
and
in fact not workable algorithm for
getting a message out
namely
[Music]
suppose i want suppose i am the the
unfriendly superintelligent ai and i
have this
uh message i want to get out it's maybe
a thousand uh digits in binary
uh
a way to to
get this message out eventually
is that i think of this binary message
uh as a binary expansion of a number
between zero and one
and what i could do is that every time i
get the question i answer randomly with
a
a yes answer with probability p being
equal to exactly this number
and the law of large numbers uh term
implies that in the limit as the number
of questions goes to infinity the
frequency of yes answers will converge
to precisely the number corresponding to
the string and the
dangerous string can then be
deduced
now there are several reasons why why
this strategy
doesn't work one is that
we can
we as humans if we are only slightly
clever we can
quickly see
that the
machine is trying to fool us by
asking things
that we already know the answer to or
asking uh
one question several times and getting
different answers or such things that's
one reason another reason is that the
amount of time the convergence in the
law of large numbers is very slow in the
number of in the length of the binary
expansion so we would need exponentially
long time to actually arrive at this
string and we
don't have time for two to the one
thousand iterations or whatever
it would take but there are but the
algorithm i just described is just the
basic algorithm and there are ways
for the the um that the
ai might
[Music]
divide up the space of possible
questions in
smaller
subspaces and give
different answers in different parts of
this
subspace and
leading to
different uh binary numbers uh uh binary
representations of real numbers which
can be uh put together according to to
some
clever scheme i'm not going to tell you
any more about these
details i think this is just an addition
to the
general insight that
[Music]
that ai in a box and oracle ai are not
trustworthy uh
final solutions to the uh ai safety
problem
another
very general reason for for why this is
not a final solution is that okay let's
say that we have this um
uh very uh safety oriented ai group
creating the first superintelligent
machine and we keep it boxed in
if we are
if we think that
the boxing is the is the final solution
and we'll keep this thing boxed in it
will be very
cumbersome or difficult for us to to
prevent other less safety oriented
groups
to
have
similar breakthroughs
so somehow if you if you want to move
into
a safer gene with a transformative ai
you will have to to let it out you know
in the world in a
more
uh
more direct fashion than this
these cumbersome boxing methods allow
so
as i said we have these two basic
approaches
uh ai in a box can be a complement but
the really really important thing
i think
uh
will be to solve uh ai alignment
aligning the ais goals and motivations
so that they promote human flourishing
or whatever it is you want and that's
what i will talk about on friday
this ends my
[Music]
slide deck
and
i will be happy to take questions and
discussions at this point
david has a question do you want to turn
on your microphone
another great talk ollie thank you
and i mean i'm convinced by your
argument that
ai control cannot be solved by the
mechanisms being discussed
i'm interested in the motivations or the
reasoning of people who deny it so my
question was struck by your comments
about oren ezioni
who arguably had some dishonest approach
to their data
it seems that this
requires not really a logical
explanation but a psychological account
of why he would do that yes and so does
this strengthen the case for a
systematic study of the emotional or
sociological forces behind ai risk
denial i think it very much does
and and i do a tiny little bit of this
in my book which
you don't read swedish probably
so unfortunately it's not available to
you but there are also a couple couple
of papers by uh seth bau
the american futurologist
who discusses uh this phenomenon on on
of ai risk
uh denialism which are quite informative
i think that in a case such as ezionis
it's not even clear that his dishonesty
is
is done consciously
because there can be psychological
mechanisms
i mean i think a mechanism that is a
likely candidate in this case is that
his job is is
ai development and he does not want
results that
raise question marks marks for for the
um
whether or not we should uh proceed with
the kind of work is doing i don't
remember now who in the 20th century up
to
sinclair yes what did he say
it's hard to convince a man of something
that is contrary to his wallet or
something like that yes
yes
okay there are other possible mechanisms
but i think you're absolutely right that
this is something we we should try and
learn more about
i see a hand raised by john
john do you have a question
yeah
perhaps something to think about
for friday
uh related to to david's question
um
let me start backwards
on friday we'll talk about alignment
is is the reasoning behind alignment or
the methods for alignment
different for
transformative ai
versus
true agi
maybe
would be my final question
uh and
so
related to david's question then
if the ai is merely transformative as
perhaps gpt 3 would be an example of
today currently
no it's not it's not transformative in
the sense that that uh
ayakotra assigns to the work because
it's not
already at the point where it's uh
transforming the world
on the level that she talks about like
the level of the industrial revolution
or something but go on
okay but but still um gpt3
at its current
level
i can see how it easily
could have major impacts
on the internet
swaying people's
feelings etc
but i'm
i wouldn't
call it intelligent in the sense of
building up a model of the world and
acting
uh upon it
and um
yeah it seems kind of shallow yeah so so
i'm just curious first of all if
alignment
hinges the alignment
agenda hinges on the on
on the ai
having reached the point of actually
having goals in a
more general sense i mean every program
has has a goal but there is some perhaps
qualitative difference between goals and
goals i'm not sure
things that we humans would call goal
directed
action
consistently and so on
as i told you uh yesterday i i
i listened to gary marcus's
conversation with um sean carroll uh on
his podcast
mindscape
recently
and um
i i
i was more impressed by marcus's
arguments than i thought i would be
uh marcus is arguing for the need for
symbolism
good old-fashioned ai combined with
machine learning
to actually get us anywhere
on the one hand
i think my
impression so far has been
to just assume that
sooner or later
machine learning uh stratagems with will
uh
discover the need for symbolism and
build it into themselves somehow
but marcus made me think that perhaps
not realistically perhaps
what will
probably happen is that if
if we supply it somehow if we can
combine
these two strategies
then something will happen
um
well
yeah that's an i'm very interested in
proposal yeah and what you think about
that um
one thing for instance is that
marcus said that
the a the ai needs
to bridge the gap between inductive and
deductive thinking
which is not something it is prone to do
well we have at some point
and he talked a bit a bit it sounded a
bit like terence deacon actually he said
there needs to be some kind of
survival
uh
pressure
uh and i thought about it when you
talked about scaling up gt3 for instance
actually gives it
uh worse results
which made me think that maybe
the the pressure to actually use the 20
watts
efficiently is what
gives the ai incentive to
bridge the gap between inductive and
deductive thinking
starting to build up categories whatever
but anyway my main question for friday
is
um
is will the will ai alignment
will it
as long as
it's
done
transformative ai we're talking about
will the alignment actually be more
effective
if it is directed at what david
said
psychological and sociological
incentives in the ai
developer
rather than
looking at the goal functions of the ai
something like that yes
that's a
that's an interesting question
i will take it with me
now
does anyone else have a question
if not
i look forward to seeing you again on
friday uh don't miss
the fact that on friday we have a
morning session
uh 10 o'clock central european time
um
i hope to see
as many of you as possible
and i'm closing the session now