#034 Eray Özkural- AGI, Simulations & Safety

welcome back to street talk
today we are talking with adai uzkuro an
agi researcher from turkey
here are some of the best bits i'm sorry
those are not logical falsies at all
they are very
very well as refutations well the
objections you're touching on
a really key concept here which is first
of course i am i am we only
tim we only touch on key content
i don't care if a computer can write a
blog post better than i can gpt3
probably can already
but i don't think of gpg3 as being
intelligent i i don't think so
i think you're smarter than gpg3 and
don't take that as a compliment
microsoft clippy can write a blog post
that's better than i
can honestly um tim is only allowed to
mention
francois chole once an episode but um
i'm giving so we actually had quite an
animating conversation today
this has been one of the most
interesting episodes we've ever done
i've
i've never seen culture so animated
before
because i feel like we've been doing
this some epsilon greedy thing
you know where 10 percent of the time we
explore a new part of the search space
and i think we finally
lit kilter up not that he's not normally
lit up but today he was lit up like a
christmas tree
so arai is a really interesting
character being an agi researcher he's
done some of the kind of
usual suspects so things like
algorithmic memory
and algorithmic probability he's
published a paper on what is it like to
be in a brain simulation
and also his omega his um architecture
that he's just released this year for
artificial general intelligence
now um arai has a very interesting blog
exa
machine.net forward slash blog and some
of the articles are extremely fruity
he talks a lot about this ai doomsday
eschatology
so calling out folks like elizia
zudakovsky and max tecmark and
nick bostrom and so on he seems
particularly aggrieved by the simulation
argument by nick bostrom he thinks it's
a kind of new age
creationism and just to be clear on that
i don't want to defame nick bostrom i
can't find any reference online to him
actually being a creationist
everyone these days seems to think that
we are on the brink
of the ai singularity even my childhood
hero john carmack this morning
posted this on twitter i must admit i'm
pretty disappointed in john
i never thought it would be possible for
me to disagree with anything that he
could
possibly say but in this case i do
disagree
as francois charles recently said in his
measure of intelligence paper
you can buy skill with infinite amounts
of
experience and priors but intelligence
is something different
intelligence in chile's view is the
ability to
generalize and also the information
conversion ratio
it doesn't necessarily mean having lots
of compute
a few weeks ago we spoke with conor
leahy about the ai fire alarm
and ai safety how can we make the world
a better place how can we ensure that
humans get what they want
and that whatever we become in the far
future the other races of the galaxy if
they exist
are proud of what we've become a
significant number of folks now
think that the ai doomsday scenario is
legitimate
it's at least a non-negligible chance a
lot of folks now seriously think that
super intelligence is just beyond the
horizon
they think that this doomsday scenario
where if there were a super intelligence
it would
take over the universe and it would want
to kill all of us without any regard
whatsoever
this is what the public intellectual sam
harris and elizia
judokovsky had to say about it obviously
we'll be getting
better and better at building narrow ai
you know go
is now along with chess seated
to the machines how do we begin to take
safety concerns
seriously enough so that we're not just
committing some slow suicide
and we're actually having a conversation
about
the implications of what we're doing
that is tracking
some semblance of these safety concerns
i have
much clearer ideas about how to go
around
tackling the technical problem than
tackling the social problem
if i look at the things that way that
things are playing out now
it seems to me like the default
prediction is
people just ignore
stuff until it is way way way too late
to start thinking about things
connor like many of the folks in the
rationalist community
think that artificial general
intelligence is already here
with gpt-3 i think gb3 is agi i think
gpt3 is as intelligent as human
and i think that actually is probably
more intelligent than a human in a
restricted way
in a very specific way i was convinced
in 2017 that the bubble has burst deep
learning is
dead like why do we even research it are
you kidding me matrix multiplications
wow intelligence boys we did it it's
you know it seemed so preposterous
because of this position
these folks think that ai safety is of
paramount importance
even now argument number one
intelligence is going to be very
powerful
argument number two instrumental
convergence happens argument number
three
defining correct utility functions is
very hard argument number four
the value functions that capture human
values are an extremely small subset
i actually think that we should not want
a robot
that will do anything we say i would
prefer that if
i told my robot go murder innocent
children the robot says no i'm not going
to do that
i want people to be happy i want
suffering to be minimized
by whatever means possible i do not give
a single
 how we achieve a better world i
just care about us achieving a better
world
this is what robert miles had to say on
ai safety
a few years ago i've been talking about
ai safety for quite a while now
a lot of ai safety advocacy has had a
certain
quality to it which to be honest i'm
sure i've contributed to a little bit
and it's true that might be the
appropriate response to the problem
but if you want to warn people about
something it matters how you phrase
things and how you frame things
ideally the arguments would stand on
their own merits people should be able
to look at the facts and decide for
themselves
but at the end of the day most people
don't have the time to put in to get the
required level of understanding
and they shouldn't have to i mean it's
not actually possible to study
everything in enough detail to
understand it there's just too much to
know
so we have experts but this kind of
sucks because then how do you choose
your experts
i have a question as you say this kind
of dooms view is very
widely spread and sometimes and this is
what
bugs me it comes out of these big places
like
google or like the big universities and
so on
what do you think that is i think some
of the founders and
first employees of deepmind were
close with the ai risk people
they supported their views if i can
convince everybody that ai is super
dangerous
right and that the only people who are
qualified
or enshrined with the responsibility of
working on it
are the big people then the big people
can control
all of the research they can control the
funding they can direct it to where they
want it to go they can
fear monger everyone into giving them
the keys to the kingdom
and it's also means to control
ai industry they want a totalitarian
solution and
this this kind of this ludditism they
want to prevent ai from being developed
by people they don't agree with
there's no fire alarm for artificial
general intelligence
what is the purpose of a fire alarm you
might think that the purpose of a fire
alarm is to tell you that there's a fire
so you can react to this new information
by getting out of the building actually
as we know from experiments on
pluralistic ignorance and bystander
apathy
if you put three people in a room and
smoke starts to come out from under the
door
the people like it only happens that
anyone reacts
around like a third of the time people
sort of like lance around
see if the other person is reacting and
they see but they like try to look calm
themselves they see other people trying
to look calm
they conclude that there's no emergency
and they keep on working in the room
even as the
starts to fill up with smoke there's no
fire alarm for artificial general
intelligence
there's all sorts of things that could
be signs
alpha zero could be a sign maybe alpha
zero is the sort of thing that happens
five years before the end of the world
i have no idea how to build an
artificial general intelligence
and this feels to them like saying that
it must be impossible and very far off
but if you look at
the lessons of history like most people
had no idea whatsoever how to build a
nuclear bomb
fermi said that a critical sustained
critical chain reaction was 50 years off
they could be done at all
two years before he personally oversaw
the building of the first pile
and if this is what it feels like to the
people who are closest to the thing
not not not the people who like find out
about the news a couple of days later
the people who have
the best idea of how to do it towards
the closest to crossing the line
then the feeling of something being far
away because you don't know how to do it
yet
is just not very informative i mean it
could be 50 years away
but even if we knew that the chance of
this happening
before 50 years was zero
that is only really consoling on the
assumption
that 50 years is enough time to figure
out how to do this safely
and to create the social and economic
conditions that could absorb
this change in human civilization i mean
the way stuart russell put it the same
guy who said you can't bring the coffee
if you're dead
is imagine that you knew for a fact that
the aliens are coming in 30 years
would you say like well that's 30 years
away like let's not do anything
no it's a big deal if you know that
there are aliens that there's
a spaceship on its way toward earth and
it's like going to get here in about 30
years at the current rate
we have no idea whether or not ai is
imminent
and i was like right that's not really a
reason to not worry now is it
i must admit eliza's logic is impeccable
even though i don't believe that any of
the technology we're looking at now will
lead to artificial general intelligence
anything could happen there are so many
stepping stones out there
which we haven't discovered yet which
could lead to artificial general
intelligence
maybe even in the next couple of years
i've been schooled on this way of
thinking by
the essential education from professor
kenneth stanley
we're actually going to release our
conversation with kenneth stanley
hopefully as a christmas special
another person famous in this space for
talking about ai safety
is professor stuart russell this is what
he had to say about it
i'm actually trying to change the
definition of ai so that we have
provably beneficial machines
and the principles are machines that are
altruistic that want to achieve only our
objectives
but that are uncertain about what those
objectives are
and we'll watch all of us to learn more
about what it is that we really want
very soon afterwards machines will have
read
everything that the human race has ever
written
and that will enable machines along with
the ability to look further ahead than
humans can as we've already seen in go
if they also have access to more
information they'll be able to make
better decisions
in the real world than we can
everything that we value is based on our
intelligence
and if we had access to a lot more
intelligence
then there's really no limit to what the
human race can do
we better be quite sure that the purpose
put into the machine
is the purpose which we really desire
i'm trying to redefine
ai to get away from this classical
notion of machines
that intelligently pursue objectives
stuart russell points out three
principles that he thinks are of
critical importance
for us to develop an ai that has
decent values those are altruism
humility and also this notion that
we can learn something about the values
of a system by its behavior
the thing is though this amounts to
something called behaviorism
this is something which arai criticizes
vociferously this third principle
i think is the one that you're probably
scratching your head you're probably
thinking well you know
i behave badly i don't want my my robot
to behave like me
right you know i sneak down in the
middle of the night and i take stuff
from the fridge i do this i do that
but the problem is we still get back to
this issue of how do we model human
preferences and so the machine has to
somehow trade off way up the preferences
of many different people and there are
different ways to do that
economists sociologists moral
philosophers have understood that and we
are actively looking
for collaboration people have heard
these arguments a lot if it's
more intelligent than us it will just
regard us as an
ant and it will have no trouble stomping
it if you take on that if you hear that
do you simply say i'm not convinced or
do you
have a positive case to make of why
we shouldn't be worried so much i don't
think
i don't think that's likely to happen
simply because of the fact that super
intelligent
agent will develop these deep
philosophical thoughts
so it will learn about life you will
understand
it probably will distinguish us
from rocks all the animals do
there is no reason why an intelligent
machine can't do that
so that that's not very realistic it's
it's more like you know
assuming that the cognitive
architecture of the agent is built in a
very
specific way of the utility maximization
agent or well strict the goal following
agent
which always tries to act in this uber
rational way
is it attacks and thinks single-mindedly
right
yes to win something to maximize is
assigned utility function and their
argument was that
when an agent does that it disregards
everything else and you're made of atoms
and it could just
use you or just destroy you without a
second thought because
that's what's programmed to do however
that's not a very intelligent way
to program an agent it's not a smart
design but what happens there is a
little paradoxical right
you are designing something with a very
stupid
objective and then complaining that it
does exactly that
so why did you design this
it like this in the first place right i
think the answer that's pretty simple
which is that
the human population is quite
incompetent and
shows that it's not only incompetent but
quite arrogant and frequently
designs and releases things into the
environment that cause
all kinds of a host of you know problems
and we've also seen
if we do consider human beings to be
intelligent we have seen
multiple examples in our past of
different groups of intelligent people
deciding that they were superior in some
way or another or they had technological
superiority and wiping out
other groups of intelligent beings that
didn't have that capability
they're saying that it is at least a
risk factor that if we create
a hyper-intelligence that there is this
risk factor that it may decide that
everybody in connecticut's just getting
in my way right now so i'm going to
annihilate them to build a larger
processor
if we can go back a bit to the the point
i just made is
i think at the very least we can all
agree there's a huge amount of
uncertainty here
people are imperfect we're imperfect at
coding things we have imperfect
motivations
yadda yadda there's a lot that we don't
know would you agree
that this is at least a risk factor that
agi
will not for sure pose no threat that it
could pose a threat and that at least
it's worth evaluating
that risk factor agree and i agree with
you that there is
a lot of doom saying and they make the
opposite assumption which is assuming
that
no matter what we do it's going to seek
to destroy us like but
is this at least a dimension that we
should be thinking about i.e
it is a risk factor i admit that
there are risks from super intelligence
or
transparent intelligences it depends on
the
design for instance if you're assuming
this
reinforcement learning agent the utility
maximizing agent
then you'll have to specify a utility
function
and you even when you try to
define benign harmless utility functions
or
ones that seem to be accomplishing some
general universal good you might end up
hurting people
that's that's quite possible but this is
a legitimate dimension
of research and thought the problem is
there seems to be a dichotomy between
the very real and present danger now
of things like robotics safety and
warehouses and
the military use of weapons and the kind
of doomsday prophecies that we hear from
nick bostrom and
eliza zulakovsky concerned yes it is it
is it is
risks from ai do exist robots have
killed factor
workers right why did it happen
because it didn't happen because the ai
was too smart
most of the ai eschatologists rely on
this kind of
intelligence explosion concept to
underlie their premise but is that even
plausible
francoise charlay doesn't seem to think
so this is what he had to say
on lex friedman's podcast but what
happens with the recursively self
improving system
is typically not explosion because no
system exists
in isolation and so tweaking one part of
the system
means that suddenly another part of the
system becomes a bottleneck
and if you look at science for instance
which is clearly recursively
self-improving
clearly your problem-solving system
scientific progress is not actually
exploding if you look at science what
you see
is the picture of a system that is
consuming
an exponentially increasing amount of
resources but
it's having a linear output in terms of
scientific progress
as you make progress you know in a given
field
or in a given subfield of science it
becomes exponentially more difficult to
make further progress
and that's exactly the picture you're
seeing with science that
the number of scientists and engineers
is in fact increasing exponentially the
amount of computational resources that
are available to science
is increasing exponentially and so on so
the resource consumption
of science is exponential but the output
in terms of progress in terms of
significance is linear
one of the fascinating concepts here is
that as you scale in one area
you get inertia or friction in another
area
exponential friction like the more
researchers you have
working on on different ideas the more
overheads you have
in terms of communication across
researchers i'm just trying
to make an argument to question the
narrative
of intelligence explosion which is quite
a dominant narrative
it is part of the identity of many
people if you go against this narrative
it's like you're attacking the identity
of people who believe in it
it's almost like saying god doesn't
exist or something right
so you do get a lot of pushback if
most people who actually have built
anything that you could call ai
quote unquote would agree with you if
you look at
the mythology of most civilizations it's
about
the world being headed towards some
final events in which the world will be
destroyed
so yannick and i have made about six
videos at this point on francois chale's
on the measure of intelligence paper i
thoroughly recommend you go and check
them out to
get more insights from the great man
himself to me
all of this stuff just seems completely
obvious
intelligence isn't general it it has
degrees of generality that is
defined by certain conditions and any
degree of generality will usually have a
concordant
deficit of generality in another domain
so
intelligence can never really be general
you know at best it can have some degree
of generality like human intelligence
it also always has some specialization
in the same way that human intelligence
is specialized
in a certain category of problems is
specialized in the human experience
and when people talk about hdr i'm never
quite sure if they are talking about
very very smart ai so smart that it's
even smaller than humans
or they're talking about human-like
intelligence because
these are different things but this
thread of research is really quite
different
it it builds on ideas this notion that
there's going to be an intelligence
explosion yeah the thing is
philosophically speaking if there is a
non-negligible chance
of an intelligence explosion then
it does make it a legitimate form of
research i'm not entirely sure
because the risks attain rates are
overblown
they said themselves there's a 20 chance
that agi will destroy the world
so there's an 80 chance that it'll save
it
right they're doing the world's most
important work
that sounds just like tv evangelists
send all your money to us and you will
be saved
sort of rhetoric and that's dubious of
course
that's not plausible to say the least i
i will get the intelligence explosion
whether it's possible or not but first
the existential risk
thing there will be risks from
autonomous asians
okay so that's real there could be a
drone
that shoots people that's a a military
drone might malfunction and
harm people or there could be an a
home robot could hurt people
the important thing to understand here
is that
the bulk of their argument is just
science fiction
it has no scientific basis whatsoever
just to frame the singularity thing it
was by this ray
kurzweil he says that once the
singularity has been reached
that machine intelligence will be
infinitely more powerful than all humans
combined
and then he just predicts that it will
radiate from there alai has an
interesting take here
he thinks that agi can be estimated as a
function of information processing
efficiency
the energy efficiency of information
extraction as
a measure of intelligence so we try to
calculate the computational bandwidth in
the neocortex
right and maximum we try to find an
upper bound and i use the information
theoretic
communication bandwidth between synapses
as a basis
because of the uh computation occurs
along the synapse in the brain that's
more important than the neuron
that that's the right number to work on
okay that gives you an estimate of four
petaflops per second for 20 watts only
that's great
energy efficiency and then i projected
using kumi's law
when we will get to that energy
efficiency and the date is exactly 20
30.
okay so that was easy it's just fitting
an exponential plot
anyone can't do that but you have to
have the right assumptions
so everyone got it we're not using
moore's law
because moore's law is dead our brains
have local processing and all sorts of
characteristics in the substrate that we
couldn't necessarily replicate with
technology
you mean architectural properties but
we're talking about intelligence we're
not talking about neurophysiological
properties we're not talking about
replicating
all the neurological properties in the
brain oh that's a valid point
which i make i hope you linked to it my
paper on agi paper on brain stimulations
like it's very important if you need the
human life
subjective experience you need to
replicate those very
properties so i ended up agreeing a
little with john soro
i hate to say that because because he
thinks computers can't do consciousness
and i think they can but human
consciousness
subjective experience might require is
very particular quantum states
we don't exactly know which ones yet but
yeah
however that's beside the point i'm
talking about human level
artificial intelligence right so i'm
talking about
ai that matches the performance of human
tasks
you cited in your blog that we'll be
able to write blogs or compose
music better than we do now and then
we need to start talking about what do
we mean by intelligence because humans
have this broad generalization
right do you know what i mean so i don't
care if a computer can write
a blog post better than i can gpt3
probably can already
but i don't think of gpg3 as being
intelligent i i don't think so
i think you're smarter than gpg3 and
don't take that as a compliment
microsoft clippy can write a blog post
that's better than i
can honestly um
yeah i was i was wondering i was
wondering about that too
this notion of human level intelligence
right what do you understand
or or how do you okay so i'm trying to
explain
so what i propose and actually this was
initially proposed by ray solmanov
himself
and he pretty much said the last word on
all these definitions
it's just that people don't really
understand it
but or don't care to read carefully but
a lot of people have imitated him
the upside is that energy efficiency of
information extraction is a very good
measure
in this case energy efficiency of
prediction
is which you can use as a basis for your
machine learning system at some level
we'll be comparing
the information extraction efficiency of
our
deep learning architectures with the
human visual system the
cortex and the visual cortex there is a
sort of an argument that we have to
and this is this goes on the lines of
chole and i i know tim is only allowed
to mention
francois chole once an episode but um
i'm giving myself down to one from
because he talks about these
we also have to take into account the
sort of process by which a system
is created and that process for the
human brain
for example is evolution right the
entire
evolution of life itself that
made these synapses the way they are
that is
quite a bit of energy in that process do
you think that
should go into a calculation or do you
think intelligence is just what
you know is at the end it's apples and
oranges
yeah because we're doing intelligent
design but evolution is an intelligent
design
what matters is what comes at the end i
don't care how long it took how much
energy it took to get
to that black box the measure of the
intelligence of that black box
should only be a function of that black
box
period that makes sense and i would just
add to that it's
the ability to solve problems
efficiently so we have to have some
measure of efficiency built like energy
efficiency
look at least in the context of agi
killing us all like one thing that
annoys me about sort of the doomsayer
side of the argument is and it's exactly
along the lines we've been talking about
the human brain
is they fail to appreciate how efficient
energy wise the human brain is for what
it does compared to our computers
and they completely leave out any such
constraints
on the agi oh we're going to build this
paper clip machine and it's just going
to be
better better at everything and then uh
you know it's just going to start
consuming the world and they just ignore
energy barriers energy limits
gravitation a material
diffusion speed of light it's like
everything just gets ignored for the
sake of
this agi that's just better at
everything right
i don't believe it's possible with any
hardware substrate to even
reproduce something that looks like the
brain is the human brain hardware
i think it is it may be mushy but it's
hardware
it is but it seems to have
characteristics that we couldn't
reproduce but at some point in a distant
future when i have a
nano printer 3d printer i could print
out
a human brain the human brain is just a
beautifully complex nano machine at the
end of the day
made up of all these little molecules
and so that is hardware
i think your question is more about can
it be reproduced with
turing computable equivalent processes
like describable processes or whatever
that i think is an open question
like an entirely open question turing
computation
can reproduce the human brain like like
uh
i said earlier there may be like these
sort of quantum or
even continuous pieces in there analog
kind of components
that matter in some way i don't know i
think that's an open
question but my point is at the end of
the day the brain
is hardware oh i agree i just mean that
we wouldn't necessarily be able to
recreate it or
manufacture it the way it was going to
be pretty
pretty damn hard but on the other hand
you know we're going down the route of
bioengineering right so i mean
like probably the first nanotechnology
that we produce is going to be a
mosquito that's programmed to deliver
poison
for like kind of an assassination tool
or something like we're going to go in
there and modify its genome
to make it behave in a certain way that
we want but
i think it's speculation but i do think
it's a risk factor that's worth
researching
i'm saying the cosmos prefers energy
efficient
processes you're saying that the cosmos
hates deep learning then
no it doesn't like deep learning is
pretty efficient
but on gpthree though yes there are
different schools of thought right
so some people believe that there's this
weird thing happening you know you're
talking about
this emergent phenomenon that if you
because the perplexity just keeps
getting better and better
you scale up this language model and all
of a sudden weird stuff
happens and this kind of intelligence
emerges
well the the some of the mary folks have
this
have this view okay so they probably
might not have a complete understanding
of deep learning
models it's uh because they would
contest
that yeah they don't understand it
though
if they had understood it they would
have known that as a
kind of a syntactic language model it's
not
well you can apply to agents of course
you can apply them a sequence prediction
model to agents but it's probably
not like a hash table but it's not using
information as efficiently as it should
it doesn't
it probably doesn't represent the world
as it should so it's probably not going
to be very intelligent it's not going to
be feasible or
efficient enough to do that i think so
the chinese room is like gp3 right it's
following some syntactic rules
searle argues that even if this room
produces all the right answers it
doesn't have understanding so that's
therefore it's
it's not conscious or it's not
intelligent or whatever source
conclusion isn't exactly right
but this is a good analogy because of
course there's a difference because
here the instruction book isn't fixed
instruction book
is learned yeah it's a machine learning
system
but the instructions depend on
only text right when we interrogated
gbg3 and we've been playing with it
a lot the overwhelming impression that i
got because we had this wonderful
gentleman called walid subba on the show
the other week
and his big thesis is that natural
language processing is not the same
thing as natural language understanding
and he gave us loads of examples
now you can get very well at learning
syntactic representations right
but this will never give you semantic
representations because they are not
about the states of the world
they are not they don't include common
sense representations
and that's the problem some linguists
notably behaviorists
okay so that's important radical
behaviors
thoughts okay that there is no
such extra thing as semantics so match
these words
in some time difference algorithm it
would give you everything but they are
wrong of course
okay so the guys who designed gpt
probably have a behaviorist mindset and
that's why you know they designed it
like that
they think it solves the entire problem
and they will probably try to scale it
up and believe it solves everything
but what it really does is just copy and
paste really nick bostrom
had this argument from simulation it
went something along the lines of
if we develop the technical
sophistication or any
if any civilization out there developed
a level of technical sophistication that
allowed them to build a computer
simulation
then there must be many more computer
simulations than there are actual
civilizations therefore we probably live
in a computer simulation something like
that
many people have heard of the
philosopher nick bostrom and his famous
argument from simulation
this is nick bostrom talking on lex
friedman's show there would be more
simulated people with our kinds of
experiences than non-simulated ones
like if in kind of
if you look at the world as a whole by
the end of time message where you just
count it up
there would be more simulated ones than
non-simulated ones
then there is an extra step to get
from that if you assume that suppose for
the sake of the argument that that's
true um how do you get from that to
the statement we are probably in a
simulation
you don't need infinite time you should
need what
how long does the time but however long
it takes i guess
for a universe to produce an intelligent
civilization that attains the technology
to run some ancestry simulation
so interestingly arai links the
simulation argument to a kind of
new age creationism and as i said before
i don't want to
defame nick bostrom i can't find any
reference to that online
but it kind of makes it a science
plausible religious
narrative this is a new one to me but
interesting nonetheless
but just to make it clear i don't think
that we should take into account any
quasi-religious motivations behind this
philosophy
there is a kind of link here to the
intelligent design concept
because if the world was simulated then
you could argue
that it is the creation as envisioned by
the creator of the simulation probably
that's not true though because anyone
who studied
neuroevolution algorithms knows that
these are
open-ended algorithms and their
stochastic algorithms they
might have some degrees of environmental
determinism but that's not a given
anyway personally i really like the
simulation argument i've always been
intrigued by it
ever since i first heard of it and he
himself even says that the simulation
argument is weaker
than another argument that he's a
proponent of the doomsday argument
there seems to be a bit of a pattern
here with bostrom in the case of the
simulation argument
is much weaker than the assumption you
need to
make the doomsday argument go through
because you could run many simulation
and also because
within the simulation civilizations
would start
that again would start simulations
that's good
exactly so you said that ancestor
simulations are silly
you say that and the simulation tech is
implausible that this stuff
would requires evidence to be taken
seriously you've even said that it's a
form of new age creationism
you've said that we should be
open-minded is not a good argument for
taking this seriously
and it's it's interesting that it's a
bad argument for not taking intelligent
design
seriously which i think you've got quite
a good point on there and you also said
that
you're not convinced that we would be
able to detect a simulation if we were
in it
as carl sagan would have said
extraordinary claims require
extraordinary evidence and
in this case we definitely don't have
that kind of evidence
you're an agi researcher right so i
assume you're optimistic that at some
point in the human future
we will be able to create agis would you
agree with that
okay so at some point at some point in
the human future whenever it is
500 years from now 2000 years from now
it doesn't matter
we ourselves like this civilization will
be able to create
simulations in which we embed
millions billions of agis and have them
experiencing a
self-consistent world that has physics
and everything like that
what would stop the agi from making the
same argument to conclude they're not
simulated
what evidence would they find that
they're living in a simulation that we
are not they would
they would find limits and artifacts
this gets into the exact arguments that
young earth creationists exactly and
they'll be like
you'd find evidence but then couldn't we
plant that evidence
as makers of the cosmos couldn't we put
that evidence there and couldn't we make
it
appear big but couldn't we put the light
already into place
racing towards earth and things like
this i agree with that
like you you can never you can never no
no no no
this is just silliness this is silliness
that has nothing to do with the
what i'm saying is i asked you in that
hypothetical scenario where i've i'm
simulating a bunch of agis
what artifacts would they observe that
would lead them to conclude they are
simulated
and the example you gave was limits and
artifacts
what artifacts would the simulated
people find
that lead them to conclude that they are
in a simulation
right so uh i think they would find that
their quantum states are somehow
not quite correct and there are some
other forces
inaccessible to them but they can see
the
effects of those forces by dark matter
and dark energy right now see this is
where this is where you and yannick go
down the
irrelevant area like the fact that
creationists
like like the simulation argument
doesn't mean that the simulation
argument isn't scientific
right right or can't be evaluated like i
don't i don't care what creationists
and i thinks that all of this is just
reality denialism
and as a result of that it leads to
faulty ethical reasoning
being incorporated into their kind of
models of the world do you think that
their ethics is faulty because
they believe in the simulation argument
because if they were so nihilistic that
they believed that they were in a
simulation
then it's almost like nothing matters
anymore and and their
thinking on many other matters would be
tainted they're actually worried about
simulation shutdown risk
these folks actually believe that
they're in a simulation
and that ai doomsday is more threatening
to us than nuclear armageddon and that's
the only thing we should be worried
about so that kind of damages their
thinking processes in other domains
of course all forms of creationism
devalues human life
and it's it's a way to deflect reality
if you're not
sophisticated enough psychologically to
accept reality as it is
then you develop these psychological
defense
mechanisms like afterlife and so the
mytho and afterlife or
heaven hell or stuff like that they're
all
connected but they're all reality denial
at a score
and they usually lead to very wrong
ethical systems so that's a big problem
and of course that's why because bostrom
is a creationist
the realm of uh science and logic here
like i don't care what people's
you know motivations they learned as a
child
okay so so can you please let me refute
robustrom and then maybe you can
maybe you have a version of that camp to
refute it but i don't think so
sure first of all it's very costly to
simulate
the world first of all you need quantum
computers
and you need this gigantic quantum
computers and
excuse me what are you going to use this
for ancestor simulations
seriously that's a mind that's mind
projection fallacy
you're projecting your kind of personal
preferences onto a future
civilization that we don't comprehend
can you please allow me to elaborate
let me explain how how damn stupid that
is
it's a it's a fallacy
it's not a okay it's not a it's no
fallacy
okay it's it's called the refutation
[Laughter]
but let me first explain it and then you
you're free to disagree
um now about the motivations of a future
civilizational posthumous it is bostrom
that makes the assumption that with
what with 50 probability they will be
making ancestor simulations so is
what kind of an assumption is that you
could have stacked simulations
yes and that could then be uncertainty
as to which level we are
at um as you remarked also
all the computations performed in a
simulation within the simulation
also have to be expanded at the level of
the simulation right
so the the computer in basement reality
where all these simulations with the
simulations where the simulations are
taking place like that that computer
ultimately it's it's
it's cpu or whatever it is like that has
to power this whole
tower to me personally the biggest
argument and i agree with arai a bit
right here is a let's say a
thermodynamical argument or a
computational argument
if we were to do what keith suggests and
simulate
a universe with let's say beings in it
and let's say we can't
simulate our own because that would
require let's say a computation
like so big so all we could do
is simulate somehow a reduced universe
like with more maybe consistent maybe no
artifacts at
all but it it's somehow reduced in its
complexity
and that means to me that means the
there can't it can't go on right the
simulators can't then simulate again and
simulate again
indefinitely because there's always this
kind of reduction at the end
the simulation just consists of one bit
of going like
nick bostrom simulations using a small
portion of its resources
it probably wouldn't be able to run
infinitely many i mean if
we take say the observed
the the physics in the observed universe
if we assume that that's
also the physics at the level of the
simulators
that would be limits to the amount of
information processing
that one civilization could perform
in its future trajectory and quantum
computation is very costly
and to simulate physics accurately you'd
need a lot of quantum computation and
you can't just
waste the precious cosmic resources for
a silly video game
that's not something a super intelligent
entity will do okay
once humans are eliminated from the
equation nobody will give a
flying f about us so that's not gonna
happen
but a more important objection is of
course
the argument from induction right when
all else are equal
right we prefer the simplest explanation
not the most complex schizophrenic
explanation so that's why we reject the
simulation argument let's suppose i'm my
future self and i spin up a simulation
and i put in some parameters and i start
simulating
a universe because i find it interesting
and i have the resources to do it and by
the way people waste resources all the
time like look at bitcoin
you know it's inferior in like every
single way transactionally
computationally whatever but we're
spending huge amounts of you know
about energy resources you know
calculating it
i spent up my calculation with
parameters and it may happen that on
some planet
life evolves i may not have cared like
that may not have been the intention of
my
simulation i may have just been trying
to explore different type of cosmologies
and it happened to be a universe that
was favorable for life and so some agis
spin up in my
simulation like whatever you know i mean
i'm just i'm just observing things
my future self is not going to simulate
our universe i'm going to simulate
other universes that i have the
resources
to simulate and my point of making here
is that agis may crop
up in that universe and they may have a
zoom call
just like this one where one of them
says like we can't possibly be simulated
because we don't have the resources to
simulate ourselves
it might be the case that we're living
in a
computational universe just as edward
franken or stephen wolfram would say
in this case the idea of a virtual
machine
might be a good model of how a universe
is formed
how it might be viewed in a more
scientific way via the
lens of evolution so if we're saying
it's evolution it's scientific
if we're thinking of some gods making
universes
it's not scientific what i'm saying is
that certain arguments
are not valid arguments against
simulation one of those arguments for
example
we don't have enough resources to
simulate our own universe is not an
argument that we're not in the
simulation
okay saying that i don't understand why
future beings would want to do
simulations is not an argument for us
not so i'm just pointing out there's a
lot of logical fallacies
on both sides of this argument uh i'm
sorry those are not
logical falsies at all they are very
well as refutations
veiled objections no no fallacies like i
can show you symbolically some other
time
oh there's just a straight up fallacy
mind projection
i'm sorry like godel tried to prove god
with model logic
is formal and it means nothing what
people are trying to prove about god
your that argument was a talent that's
right exactly
things even got a little bit fruity at
the end okay so so see
i'm sorry if i offended you in any way
okay like
look um what was this i saw the other
day somebody has this great quote which
is they say that
nerds are people who think the purpose
of communication
is to submit their ideas for peer review
whereas whereas everyone else thinks the
purpose of communication
is to negotiate agreement
i i i love i love this like i mean
you know this is uh this is why i'm on
this you know podcast frankly so no i
have to say
it's i have to say two things one i have
to say this conversation today was
completely not what i was expecting it
exceeded my expectations in every way
i had a lot of fun talking to you i
think you and i would actually get along
great in real life
like hang out at conferences and have a
lot of fun so
i hope we can meet someday when corona
is under control
anyway i really hope you enjoy the show
today we've had so much fun making it
remember to like comment and subscribe
we love reading your comments and we'll
see you back
next week are you guys getting paid on
the internet
okay yannick is famous
how do you do that man teach me about it
strong opinions
yeah pray to the simulator please the
algorithm
pale algorithm welcome back to the
machine learning street talk
youtube channel and podcast today we
have an incredibly special guest
we have dr arai uskaro an agi researcher
from turkey
and he's the founder of celestial
intellect cybernetics
now alai is extremely critical of max
tegmark and nick bostrom and miri
founder eliza zudakovsky
and their views in particular that ai
must be an agent and autonomous that
human preferences can be coherent
and that we should program values not
preferences
also the view that ai will necessarily
have harmful drives
and this anthropocentric view meaning
that humans are special
in their conception what area thinks of
as a reductionist form of ethics a
pleasure
and pain and that ai is more dangerous
than nuclear war
and i thinks that these views represent
a form of neoludditism
and that they're capturing valuable
research budgets with doomsday
fear-mongering
and effectively want to prevent ai from
being developed
by those that they don't agree with adai
is also skeptical of the
intelligence explosion hypothesis he
cites google as an example of
transcendent intelligence
do we see them as an existential risk
he's also extremely critical of nick
bostrom's argument from simulation
going as far as to label it new age
creationism
just to be clear we don't personally
share adai's views on many of these
topics and we even had
conor leahy on the show a few weeks ago
passionately representing the other side
of the argument
but we strive for balance here on street
talk
and if there's enough uh demand demand
for it we can actually get connor and
uri to have a conversation about this
in a few weeks time but anyway this is
our street talk this evening and
i welcome to the show thanks for having
me on the show tim
you've got a very distinctive style
you've got an interesting online blog
and
and you've really been going for the
jugular on many of these issues so
why don't you give us a framework of
understanding that explains how you have
arrived at this position
first of all i might be biased because
i'm an
agi researcher and naturally i want the
agi field to succeed
both as research scientifically and
commercially
now if people represent
agi research as an existential risk this
would
cast doubt on the legitimacy of our
research
and probably cut our funding eventually
there has been talks about that european
union
is i think now it is a bit concerned
with ai research
we might partially attribute this to the
uh skepticism raised by
ai doomsayers
what i'm doing is building
an ai toolkit a data science automation
platform
that uses some of my own contributions
and others contributions
to solve typical problems in uh
the data science field using
uh universal induction and algorithmic
memory
and some other methods like metal
learning
and this kind of system since it uses
the most general kind of
machine learning it can actually achieve
human level and can probably transcend
it
now imagine this happening then people
will freak out and
complain it's going to destroy us of
course i don't want that happening
on the other hand on my blog i think uh
people want to check it out
log.examination.net
i've written several essays about this
issue but it's not my focus i
i usually write essays on a variety of
topics related to diptech ai crypto
et cetera but i i felt
there wasn't enough response to the real
widely promoted
skepticism about agr research
so that that's why and of course these
efforts have been
very well funded and they have so much
publicity and
pretty much everyone seems to agree with
them
so i i i don't represent the agi
community just one researcher in in the
community but
i believe a lot of people in the ai
community have
given similar objections as i have
but this just hasn't been heard uh
enough
okay i have a question as you say this
kind of dooms view is very
widely spread and sometimes and this is
what
bugs me it comes out of these big places
like
google or like the big universities and
so on
and one might think if anyone
has an interest in ai or
the or agi or the field being viewed as
good and
positive and everything's fine it's not
going to kill us for sure
it is these places but yet we see
this criticism being supported or even
coming out of these places
what do you think that is why do you
think the the biggest and
more most prominent players are
tend to be more on the doom side that's
a good question
i i think it's a little sociological
because
i think some of the founders and first
employees of
deepmind were close with the
ai risk people they supported their
reviews
okay so some agi researchers do agree
that
agi presents a monumental risk
among these would be marcus sutter i
think he cited them in one paper and
supported them indirectly and there is
of course omahandra's paper on
how ai will necessarily have drives that
are
dangerous another possible reason for
that yannick is that they want to
control
ai research so if i can convince
everybody that ai
is super dangerous right and that the
only people who are qualified
or enshrined with the responsibility of
working on it
are the big people then the big people
can control
all of the research they can control the
funding they can direct it to where they
want it to go they can
fear monger everyone into giving them
the keys to the kingdom that's
one possible you know outcome of that
exactly kate that's absolutely plausible
and i think that might have happened
because elon musk was talking about this
and he funded them because i think it's
first of all free of free publicity for
them and it's also means
to control uh ai industry so if if
you're building an
agi system by investing like billion
dollars in open ai
or something like pretty much unlimited
resources for them
like they get all the compute they need
all the researchers they need everything
and then say oh hey agi people
slow down what you're doing is dangerous
or tell the government hey we need
regulations to control this
ai research because it's as dangerous as
nuclear weapons
and then so it's a similar concept to
facebook now wants regulation yannick
made this point
exactly a social dilemma because it
basically blocks anyone from entering
the market because the bar is so high
but i want to dig into this a little bit
because because when you were talking
about miri
you said that there are three things
that they propose
they think there's going to be a scary
global takeover they want a totalitarian
solution and
this this kind of this ludditism they
want to prevent ai from being developed
by people they don't agree with
now you've said that you think there's a
lot of academic pretension behind what
they do
and one of the thought experiments i
think that came from miria you cited
this is the rockos
basilisk that the ai might punish you in
the future so if you're not
good now the and um conor told us about
this story
he said imagine there's a dragon in the
future do you think that we should be
taking these arguments seriously
not too serious because these follow-up
pattern of thoughts a particular
approach
we call intuition pumps in philosophy
so uh daniel bennett talked about this
and it's it's not great philosophy
because
it it works by giving you some kind of
common sense metaphor for a hard
philosophical problem
and then shows you an easy solution
but it has all you know ugly
philosophical complexity of the matter
so that's what's happening like is that
santa claus or
i don't know like a heaven or health
story in the bible
it sounds like that but it's probably
not not a very accurate metaphor
for what's likely to happen people have
heard these arguments a lot ai
if it's more intelligent than us it will
just regard us
as a an ant and it will have no trouble
stomping it if you take on that if you
hear that
or you simply do you simply say i'm not
convinced
or do you have a positive case to make
of
why we shouldn't be worried so much
let me put it this way think about a
sociopathic person
you think that person is super
intelligent
would you assume that person is
cognitively perfect has no impediments
probably lacking in social intelligence
so
i don't think i don't think that's
likely to happen
simply because of the fact that super
intelligent agent
will develop these deep philosophical
thoughts
so it will learn about life you'll
understand
it's probably will distinguish us from
rocks
now hopefully that is better all the
animals do
there is no reason why an intelligent
machine can't do that
so that's not very realistic it's it's
more assuming
that the cognitive architecture of the
agent is built in a very specific way of
the utility maximization agent or
well strict the goal following agent
which always tries to act in this
uber rational way is it attacks and
thinks
single-mindedly right yes to win
something to maximize is assigned
utility function and their argument was
that
when an agent does that it disregards
everything else and you're made of atoms
and it could just
use you or just destroy you without a
second thought because
that's what's programmed to do however
that's not a very intelligent way
to program an agent it's not a smart
design but what happens there is a
little paradoxical right
you are designing something with a very
stupid
objective and then complaining that it
does exactly that
so why did you design this
it like this in the first place right i
think the answer
that's pretty simple which is that the
human population is quite
incompetent and shows that it's not only
incompetent but quite arrogant and
frequently
designs and releases things into the
environment that cause
all kinds of a host of you know problems
and we've also seen
if we do consider human beings to be
intelligent we have seen multiple
examples in our past of different groups
of intelligent people
deciding that they were superior in some
way or another or they had technological
superiority and wiping out
other groups of intelligent beings that
didn't that didn't have that capability
and the sort of ant analogy we don't go
out of our way to
annihilate all ants because either they
don't cause us enough trouble to warrant
that
or they may actually be useful to us in
some way or another but on the other
hand
we do extinguish millions of ant
colonies
throughout our history because they get
in the way so i think
i don't think it's fair to characterize
them as saying as those on the side of
warning about agi is saying that that
this is inevitably going to happen but
they're saying that it is at least a
risk factor
that if we create a hyper-intelligence
that there is this risk factor that it
may decide that
everybody in connecticut's just like
getting in my way right now so i'm going
to annihilate them to build a little
larger processor could we introduce a
couple of things here because
i think uh eliza zudakovsky came up with
this concept of
coherent extrapolated volition and when
we spoke of conor the other week
we started off by talking about
azimuth's free robot laws
as in that can't harm humans and so on
and those are
deontological rules so they are
basically rules saying
you have to do this you have to do that
and these folks have said that's just
philosophically weak
we need to be thinking about the
outcomes so they gave all of these
examples of how we can program values
into these agents but then they pointed
out
these problems with many of our values
and and wishes
are incoherent and inco inconsistent so
for example
i might prefer london to berlin and
paris to
berlin and london to paris and so on and
i would just be going around in circles
the whole time because
what i want as an agent is incoherent
so what's your take on that the reason
we have
philosophy of ethics is because
humans aren't very good at ethics not
everyone has a good theory of ethics in
his mind
and most people wouldn't choose to
follow those
theories anyway they would instead act
selfishly
and then there's a problem that human
values are
widely contradictory and inconsistent
from day to day or among groups of
people
like you have christian values and then
you have scientists and
you have soldiers they all have
different values and
countries nations ethnic groups
yes there's a lot of reason for war
i think this is one of the points though
that we are we have so many bugs in our
software
we seem to be quite selfish for example
the tragedy of the commons or even the
prisoners dilemma
seem to demonstrate this quite well so
if we trust ourselves to do what we
think is the right thing to do
i think we can all agree that wouldn't
necessarily be for the collective good
but
then you start getting into this
philosophical discussion about
utilitarianism for example
what is the collective good and how do
you measure it
one shortcoming of the mire approach to
ai ethics is that they seem to assume a
form of utilitarianism
however utilitarianism is not
universally
agreed uh upon by philosophers
there are famous objections to it such
as the utility monster
thought experiment and of course
what they're really uh talking about
with these ai monsters
is exactly the utility monster what is
the utility monster thought experiment
i'm very intrigued i love the name
i want to hear more right right right
util to monster is
you define some kind of utility but then
the monster
does it by killing people or that sort
of thing
there are many variations but it's
essential the same thing right
the details are not too important but
these objections have been known
and the main reason is just as tim tim
said
that's actually a very strong objection
to utilitarianism
this doesn't tell us anything without
defining the
collective good of the society or the
world you have to first
define an admissible utility function
and then maybe we can talk about it but
you didn't define it you just said okay
any utility function goes
that's not really a great explanation
it's underspecified
if we can go back a bit to the the point
i just made is
i think at the very least we can all
agree there's a huge amount of
uncertainty here
people are imperfect we're imperfect at
coding things we have
imperfect motivations yadda yadda
there's a lot that we don't know
would you agree that this is at least a
risk factor
that agi will not for sure
pose no threat that it could pose a
threat and that at least it's worth
evaluating
that risk factor agree and i agree with
you that there is
a lot of doom saying and they make the
opposite assumption which is assuming
that
no matter what we do it's going to seek
to destroy us like but
is this at least a dimension that we
should be thinking about i.e
it is a risk factor that's hard to
explain but
i admit that there are risks
from super intelligence or transcription
intelligences in my godseed benevolence
or
malevolence article the book chapter
you can find the preprint on archive and
in that paper i
i say that uh it depends on the
design for instance if you're assuming
this
reinforcement learning agent the utility
maximizing agent
then you'll have to specify a utility
function
and you even when you try to
define benign harmless utility functions
or
ones that seem to be accomplishing some
general universal good you might end up
hurting people
that's that's quite possible but there
are easy solutions to that
first of all and second that's not the
only kind of
intelligent agent i'm implying that
there might be a more
natural way to describe autonomous
agents
of course you don't need autonomous
agents at all for most tasks
like for labor automation you don't
really need it maybe for
sending a probe around mars you do need
autonomy but
it full autonomy you don't really need
it
full autonomy is when the agent has to
work
strictly without pro supervision for his
entire lifetime
it never has to be checked upon okay
that's very nice
it's like an animal but you don't need
such a design for most tasks
and you can fail in many ways and
create dangerous intelligences that's
also true but
of course we would be testing them and
seeing if they are it's not like we are
shooting random darts
okay but fair enough but this is a
legitimate dimension
of research and thought and concerned
it is it is it is risks from ai do exist
as i explained in that paper robots have
killed factor
workers right why did it happen
because it didn't happen because the ai
was too smart
there seems to be a couple of threads
here though because robots in factories
harming people that's one thread another
thread that we spoke about recently is
ai ethics
that's a very real risk now that we have
and there's lots of problems around bias
and discrimination
and fairness and so on but this thread
of research is really quite different
it it builds on ideas this notion that
there's going to be an intelligence
explosion and stuff like that
but the thing is philosophically
speaking if there is a non-negligible
chance
of an intelligence explosion then
it does make it a legitimate form of
research i'm not entirely sure
because the risks attain rates are
overblown
they said themselves there's a 20 chance
that agi will destroy the world
so there's an 80 percent chance that
it'll save it
right they're doing the world's most
important work
that sounds just like tv evangelists
send all your money to us and you will
be saved
sort of rhetoric and that's dubious of
course
that's not plausible to say the least
but to
suggest that the ai existential risk
is not negligible i don't know it might
be negligible
you don't have a scientific model do you
now right
what is not negligible like one in a
trillion
one in 100 trillions how much is that
good because you cited the example of
google as being a form of externalized
intelligence and
francois charlay makes similar arguments
about there are so many
environmental rate limiting steps to
intelligence and that the
rate of increase in science and many um
institutions and so on is actually
linear or logarithmic
it's not actually increasing yes i i
will get the intelligence explosion
whether it's possible or not but
first the existential risk thing there
will be
risks from autonomous asians okay so
that's real
there could be a drone that shoots
people
that's a a military drone might
malfunction and
harm people or there could be a
home robot could hurt people or an ai
could
decide to spam people i don't know like
there could be many
kinds of risks but when does this get to
a scenario like the popular fox theme
show or next i don't know if you've seen
it so
it's about this part a very scenario ai
breaks out of the lab and starts
scheming instantly trying to kill people
skimming it against his craters trying
to hide his intelligence
all sorts of deviant and dramatic
behavior
but that's just fiction the important
thing to understand
here is that the bulk of their argument
is just science fiction it has no
scientific basis whatsoever that's
important to understand because
um hunter didn't accept the cider his
paper is also
science fiction it's not a scientific
model it's just his idea
it's just your opinion man like it's it
doesn't really
relate uh strictly to what we're doing
however if you try to quantify the
risks from artificial intel we probably
see that
this is nothing to write home about but
such work has
might happened is it timely and relevant
does this depend on
intelligence explosion hypothesis that's
in other matters
i think that's important too so let's
get to that
intelligence explosion or what i juggle
said or
what's race almost said when it proposed
infinity point hypothesis so
actually that's the popular form that
was later popularized by kurzweil as
the singularity now singularity means
the already exponential progress in
computer
hardware technology is going to double
up and
it will go into this double exponential
regime
its acceleration will accelerate so in
any a very
short amount of time will get infinite
computing power
man that's amazing right and it says
this is also going to happen to other
kinds of
tech information technology related
fields
and this is going to result in
20 centuries of progress in one century
basically
so it's also going to happen to things
like communication
nanotechnology biotechnology will become
immortal
sorry but those are the upsides
could there be downsides to such rapid
technological progress
and should we worry about that i would
say yes and yes
we should definitely worry about the
cable capabilities posed by such
high technology that we're not going to
be familiar with
we're not going to be able to under
understand most of it
there's a cluster of philosophical ideas
that we're uncovering here
what interests me is that i find some of
them intuitive and some of them not so
this whole singularity thing i actually
completely disagree with that
whereas the simulation argument i'm a
little bit more receptive to but
just to frame the singularity thing it
was by this ray
kurzweil and he said that you described
this law of
accelerating returns which predicts an
exponential increase in technologies
like computers and genetics and
nanotechnology and
he says that once the singularity has
been reached that machine intelligence
will be infinitely more power
powerful than all humans combined and
then he just predicts that it would
radiate from there but then
our friend charles said that with these
recursively self-improving systems
because of
contingent bottlenecks and diminished
returns and counter-reactions arising
from the broader context in which they
exist
we can't possibly have exponential
progress in practice
empirically they display linear
improvement
and he basically thinks that this whole
thing is just
not possible i was actually the first to
say that
i predicted human level ai to occur
by 2030 in 2013
i think and a year later kurzweil
updated his
prediction to 2029 so we kind of agree
on that
okay and
but i was the first to predict
accurately the 2030 date
first i'll tell you how i predict that
right that's important
and that's cool okay so we try to
calculate the computational bandwidth in
the neocortex
right a maximum we try to find an
upper bound and i use the information
theoretic
communication bandwidth between synapses
as a basis
because they are computational occurs
along the synapse in the brain that's
more important than the neuron
that that's the right number to work on
okay that gives you an estimate of four
petaflops per second for 20 watts only
so that's
great energy efficiency and then i
projected using kumi's law
when we will get to that energy
efficiency and the date is exactly 2030.
okay so that was easy it's just fitting
an exponential plot
anyone can to do that but you have to
have the right assumptions
so everyone got it we're not using
moore's law because moore's law is dead
but it isn't one potential problem here
that
we are equating levels of intelligence
and
even with consciousness we get onto
chalmers and and the the emergent
properties of consciousness then you
call it the hard problem but but
what we're doing is we're taking a
metric and we're projecting it forwards
and we're saying
okay i think in 2030 that we'll have
some kind of intelligence developed
that that doesn't seem robust to me it
doesn't seem robust
but what if our algorithms are as
efficient as brains
then it would make sense right the
efficiency
is a property but of course it is our
brains have
local processing and all sorts of
characteristics in the substrate that we
couldn't necessarily replicate with
technology
you mean architectural properties but
we're talking about intelligence we're
not talking about neurophysiological
properties we're not talking about
replicating
all the neurological properties in the
brain oh that's a valid point
which i make i hope you linked to it my
paper on agi paper on brain stimulations
like it's very important if you need the
human life
subjective experience you need to
replicate those very
properties so i ended up agreeing a
little with john sorrell
i hate to say that because because he
thinks computers can't do consciousness
and i think they can but human
consciousness
subjective experience might require this
very particular quantum states
we don't exactly know which ones yet but
yeah
however that's beside the point i'm
talking about human level
artificial intelligence right so i'm
talking about
ai that matches the performance of human
tasks
but on that this is quite interesting
though because you cited in your blog
that
we'll be able to write blogs or compose
music better than we do now
and then we need to start talking about
what do we mean
by intelligence because humans have this
broad generalization
right you know what i mean so i don't
care if a computer can write
a blog post better than i can gpt3
probably can already
but i don't think of gpg3 as being
intelligent i i don't think so
i think you're smarter than gpg3 and
don't take it as a compliment
microsoft clippy can write a blog post
that's better than i can
honestly um yeah i was i was wondering
i was wondering about that too that this
human this notion of human level
intelligence right what do you
understand or how do you okay so i'm
trying to explain
so what i propose actually this was
initially proposed by ray solmanov
himself
and he pretty much said the last word on
all these definitions
it's just that people don't really
understand it
but or don't care to read carefully but
a lot of people have been imitating but
the upshot is that energy efficiency of
information extraction is a very good
measure
in this case energy efficiency of
prediction
is which you can use as a basis for your
machine learning system is that the same
is that the same as saying an
information conversion ratio
because you're saying energy efficiency
but you mean information efficiency
energy efficiency of extracting a
certain number of bits from a source
that would give you a measure or but
that's that sounds like you're talking
about a kind of transfer
right that what about how efficiently
you represent them or what do you do
what do you do with that information
when you have it oh so
it actually applies all sorts of machine
learning problems like coding
prediction transduction it does like
you just use energy efficiency as a
performance measure here
so how much your physical machine
actually spans to do that
it's not rocket science it's pretty easy
like uh
in parallel computing we use a wall
clock right
it's it sounds like you're you're
describing the what not the how
what you're doing is you're describing a
possible future system and what
characteristics it will have so i'm
assuming that somehow
deep learning researchers would be able
to build this system that's
like a brain simulation was very
efficient and so
their algorithms would be as efficient
as the brain and
it would have the same capabilities okay
so we don't have that yet
i know like i'm assuming that they will
be able to
buy them if i understand correctly i i
think you're a huge critic of the
simulation hypothesis but you
i don't know whether it's a
contradiction but you i think you were
just saying that
you do think it would be possible for us
to simulate the brain
or of course so that's what the human
brain project is about
it's not contradicting what you've said
about the the simulation hypothesis
i didn't get to that yet okay now can i
interject
for maybe for my understanding yes when
you have two slightly different things
one is the energy efficiency of
information extraction and that's how
really how you
define let's say human level
intelligence
and then on the other hand you gave us
an example of the brain does
this many petaflops with this many watts
right
do i understand correctly that this is a
a proxy
for the actual quantity a proxy that you
used
to extrapolate best worst case
uh analysis i i tried to predict
what the maximum amount of compete would
need to simulate the uh neocortex with a
very efficient
neuromorphic architecture okay and
so that's i'm assuming someone will like
myself
will get around to writing the code
okay and then we'll have this gazillion
neural
neuron neural network that does exactly
as a human brain can
and then we'll run it on this four
petaflop computer
and it will run perfectly yeah and only
only spending 20 watts which means it
can fit on your
cell phone yeah no it was just for my
understanding because
of course just because we get the energy
efficiency doesn't
yet not mean we know how to connect the
synapses but you're
if you take your true measure of
information extraction
if i can extract information then i that
means i i have the correct synapses so
at some level we'll be comparing
the information extraction efficiency of
our
deep learning architectures with the
human visual system
the cortex and the visual cortex and
we'll compare this and we'll find that
okay we're doing as well
at some points hopefully we're probably
not there yet
there is a sort of an argument that
we have to and this is this goes on the
lines of charley and i
i know tim is only allowed to mention
francois chole once an episode
but um i'm giving my job down to one
from
because he talks about these we also
have to take into account the sort of
process by which a system is created
and that process for the human brain for
example is
evolution right the entire evolution
of life itself that made these synapses
the way they are
that is that is quite a bit of energy
in that process do you think that should
go into a calculation or do you think
intelligence is just what you know is at
the end
it's apples and oranges yeah because
we're doing intelligent design but
evolution is an intelligent design
if i can just add something to that
yannick at least in my opinion and i've
argued this multiple times what matters
is what comes at the end
i don't care how long it took how much
energy it took to get to that black box
the measure of the intelligence of that
black box should only be a function of
that black box
period that makes sense yeah right is
that contradicting this idea that
some people say intelligence is a
function of prize and experience right
so you can
buy it if you add infinite amounts of
experience and
isn't that a similar i think what
yannick's getting at here is that
it's not a very good intelligence if you
have to evolve it for billions of years
we're gonna have to define intelligence
now uh won't we
okay so the best intelligence the
definition of intelligence i came up
with
is intelligence is able to solve
problems
are you are you more of a hutter it's
hutter changed his mind later but
that was saucer's initial description i
think i suppose
because he changed it to be something to
do with an agent solving tasks in a
variety of environments but presumably
you're now
dead against that i'm not against it
that's true
for oh asians that are like you animals
i think and i would just add to that
it's the ability to solve problems
efficiently so we have to have some
measure of efficiency built
like energy efficiency okay so yeah
because because look
at least in the context of agi killing
us all like one thing that annoys me
about sort of the doomsayer
side of the argument is and it's exactly
along the lines we've been talking about
the human brain is they fail to
appreciate how
efficient energy wise the human brain is
for what it does compared to our
computers and they completely leave out
any such
constraints on the agi so we're going to
build this paper clip machine and it's
just going to be better
better at everything and then uh you
know it's just going to start consuming
the world
and they just ignore energy barriers
energy limits gravitation
a material diffusion speed of light it's
like everything just
gets ignored for the sake of this agi
that's just
better at everything right but can we
quickly bring the lottery
and the hardware lottery into this
because i don't believe
it's possible with any hardware
substrate to even
reproduce something that looks like the
brain
is the human brain hardware i think it
is it may be mushy but it's hardware
it is but it seems to have
characteristics that we couldn't
reproduce but at some point in the
distant future when i have a
nano printer 3d printer i could print
out
a human brain the human brain is just a
beautifully complex nano machine at the
end of the day
made up of all these little molecules
and so that is hardware
i i think your question is more about
can it be reproduced with
turing computable equivalent processes
like describable processes or whatever
that i think is an open question
like an entirely open question i believe
it can be
but the brain seems to have some
properties you know we spoke about
capsule networks they're very
interesting theoretically but at the end
of the day
it's hideously inefficient and it's a
sequential computation
it's a similar thing maybe an rnn is
turing complete but
god help you if you want to do anything
intelligent with it first of all rnns
are not
rnns and the way in which probably most
viewers or listeners of this channel are
not turing complete you have to add on
to it a tape and some type of processing
that has
infinite memory like just a typical rnn
is not turning complete
it has to be within a little bit of a
shell there that has some extra stuff
but
i think i'm not as confident as you that
turing computation
can reproduce the human brain like like
i said earlier there may be like these
sort of quantum
or even continuous pieces in there
analog
kind of components that matter in some
way i don't know
i think that's an open question but my
point is at the end of the day
the brain is hardware oh i agree i just
mean that we wouldn't necessarily be
able to recreate it or
manufacture it the way it is well it's
going to be pretty
pretty damn hard but on the other hand
you know we're going down the route of
bioengineering right so i mean
like probably the first nanotechnology
that we produce is going to be a
mosquito that that's programmed to
deliver poison
for like kind of an assassination tool
or something like we're going to go in
there and modify its genome
to make it behave in a certain way that
we want and so at some point we're going
to be printing we're already printing
dna for that matter but at some point
our
biotechnology and our nanotechnology
will advance to the point where
we'll probably be able to create kind of
brains
structured in a way that we want them so
this sort of synthetic intelligence
if you will but if if we did that
wouldn't we have this problem we were
just talking about with the uh the
volition
right the um what alignment oh wow so we
wouldn't make it we wouldn't be able to
get it to do what we wanted
to do then then we shoot it so i mean
that's the thing is we have to you know
we have to keep it
we have to keep in mind the constraints
there which is you know
the kind of amazing thing about
sapiens or humans is that
where these self-replicating things
whose programming and recipes are
represented you know
like astonishingly small set of genes
and
and gene it's like a very compressed
highly efficient
sort of system and we function in this
environment called earth
pretty well not in every place on earth
but certainly
and so to build just imagine yourself as
a nano machine
builder and yeah i don't know you get
hired by the dod to produce
a better soldier and you go out and you
try to make one
you start facing all these constraints
well how's it going to get energy like
am i going to have a battery is it going
to be solar powered is it going to be
this or that
uh how's it going to be able to move
around should it be bipedal like or
or tripedal and you start kind of going
down this route of designing the better
soldier i'm not sure it's going to be
that radically different
from a human and then if you want to be
self-replicating
now what because all those kinds of
machines we imagine have to have a big
factory that makes them right and churns
them out but what if it has to be
self-replicating
and so even if we build something that
doesn't align with us and we go to war
with it
i'm not absolutely convinced that we're
going to lose because we have kind of a
big head start
like in this this environment but i
think it's speculation but i do think
it's a risk factor that's worth
researching
oh okay sorry certain surgic but i think
keith might be interested in this
you might want to check out my paper on
physical completeness
so that's asia agi 2015 right
the paper recital is ultimate
intelligence part one physical
completeness and objectivity of
induction i sent you the archive link
i earned the kurzweil a best agi
idea prize that year with that and
because i
introduced the energy prior
so that's the physical extension of
solomon of induction it's it's like a
physicalist
reformulation of algorithmic information
theory i i think you'll be interested in
that because
there i introduce the energy-based
definition of information so
that's as information is defined as the
length of the shortest
program but now
length of the shortest program that
produces x is the absolute algorithmic
information content of x
right now we redefine it as the
energy required to produce
x the minimum energy required to produce
or
the minimum physical volume required to
do that
that physicalist definition changes
things a little and makes the fear of
induction more
objective as you know it's
relative to a reference machine so it
solves that problem
and the energy prior is a good
alternative
to schumer to bear speed prior
where he says the cosmos prefers fast
processes
i'm saying the cosmos prefers energy
efficient processes
i i've defined all the ait
related concepts there in these new
terms
in that paper and the follow-up part two
it's on archive you can check it out
so it's something i'm still working on
you're saying that the cosmos hates deep
learning then no it doesn't like
deep learning is pretty efficient well
you should try to write you should try
to implement living search then
it's a lot more inefficient because it
tries to traverse the whole
exponential search space of programs
stochastic gradient descent is quite
efficient
actually we haven't found something much
more efficient than that that's
approximately complete for such a large
space of programs
the human brain is probably not that
different okay
the next step is probably from
theoretical neuroscience where we're
applying stuff like
the free energy principle right so there
are
alternative algorithms maybe a little
faster i don't know
maybe better paralyzable so that's
possible
but stochastic gradient descent is
pretty efficient
it's just that the problem is hard it's
a it has some you know irreducible
complexity for sure
and you're right that it's not searching
for the entire space of programs
okay it's true that rnn by itself
with no extensions memory extensions is
not equivalent to turing machine so
that's true
probably in our brains it happens by way
of cognitive architecture
like some parts of the brain work as
memory
etc and that's how we get touring
completeness at a high level of
operation what do you think about this
idea
that neural networks are glorified hash
tables it'd be interesting to get your
take on gbt3 for example
you can say that uh
any program can be simulated by a finite
state machine it's that sort of
arguments
the answer i can see where this is going
it it's a lot more efficient right so
it's when you try to pre represent
things with
static lookup tables or finite state
machines
you need a huge blow up in the number of
states so that's
usually not feasible except for the most
trivial programs
so that's what you know fear of
complexity tells us
actually that's why we need
uh universal computers right or or the
computably innumerable sets
the largest class complex class because
it's a lot more efficient in terms of
program size
and the resources used the universal
computer
can reuse all the memory states again
and again that's very efficient
but on gbt3 though there are different
schools of thought right
so some people believe that there's this
weird thing happening you know you're
talking about this emergent phenomenon
that if you because the perplexity just
keeps getting better and better
you scale up this language model and all
of a sudden weird stuff
happens and this kind of intelligence
emerges
some of the neary folks have this have
this view okay so
they probably might not have a complete
understanding of deep learning
models it's uh because they would
contest that
yeah they don't understand it though if
they had understood it
they would have known that it's a kind
of a syntactic
language model it's it's not well you
can apply it to agents of course you can
apply them a sequence prediction
model to agents but it's probably
not like a hash table but it's not using
information as efficiently as it should
it doesn't
it probably doesn't represent the world
as it should so it's probably not going
to be very intelligent it's not going to
be feasible or
efficient enough to do that i think are
you making a distinction between sort of
syntax and and semantic structures or
understanding or what
right right right can can you go into
that in a bit please
well yeah uh are we familiar with uh
searle's
chinese room experiments let's for for
the benefit of all
let's just and for my day so
of course any uh challenge experience
someone builds this you know
room with an instruction manual there's
a person in it and there's an
instruction manual
a long book of tables and instructions
that lets
someone who's given a chinese script
answer it or translate it to do some
language processing on it
by following the instructions in the
book so the chinese room is like gpt3
right
it's following some syntactic rules
and searle argues that even if
this room produces all the right answers
it doesn't have understanding so that's
therefore it's it's not conscious or
it's not intelligent or whatever
source conclusion isn't exactly right
but this is a good analogy because
of course there's a difference because
here the instruction book isn't fixed
instruction book is learned yeah it's a
machine learning system
but the instructions depend on
only text right i think that's important
you're touching on a really key concept
here which is of course i am
this notion of understanding tim we only
touch on key
content i i know we have previous for it
we have previous for it
but when we interrogated gpt3 and we've
been playing with it
a lot the overwhelming impression that i
got because we had this wonderful
gentleman called walid sabba on the show
the other week
and his big thesis is that natural
language processing is not the same
thing as natural language understanding
and he gave us loads of examples
the corner table wants a beer gpt-3
doesn't know that the corner table is a
person
or how many feet fit in a shoe gpt3 does
not know no it doesn't know that
no but what it does it tries to predict
the syntactic
language use an average writer would
make okay so i think we all agree on
that then
but do you think that deep learning
models can be scaled
to have understanding what's missing for
them to understand
okay so you can't make a bigger gp23 and
expect to understand anything
it will never do that on this sorry what
does it mean to understand
what this is well wait before we get to
understand can we first address your
first question tim which is
i don't want to go away too much but
yeah no but what
what does gpt what does gpt need
in order to quote unquote understand
while it makes the argument that
it's missing a lot of data we can't just
give it text we have to give it
semantic structure and invariance models
and things like that
so in your view what will we need to add
to
the gpt system plus the corpus that
examines what do we need to add to it in
order for it to
learn understanding to learn semantics
so the
transformer model it's just some
sequence transduction model
it's not necessary it's not an essential
part of the system but
you could use any sequence prediction
model basically
anything that embeds words or phrases
the key might be uh
learning representations now you can get
very well
at learning syntactic representations
right but this will never
give you semantic representations
because they are not about the states of
the world
they are not they don't include common
sense representations
and that's problem some linguists
notably behaviorists
okay so that's important radical
behaviors
thoughts okay that there is no
such extra thing as semantics so once
somehow match these words in some time
difference algorithm
it would give you everything but they
are wrong of course okay
so the guys who design gpt probably have
a behaviorist mindset and that's why
you know they designed it like that they
think it
solves the entire problem and they will
probably try to scale it up and
believe it solves everything but what it
really does is regurgitate
the entire corpus of human in general is
it's just
it's just copy and paste really
i don't think they necessarily designed
it it's really interesting how it's
evolved
originally language models were just a
trick that the self-supervised training
trick
and now it's just morphed into this gbt3
and it does have some really interesting
properties especially the few shot
learning
the prompt engineering you can you only
have to train it once and you can put a
prompt in there and it will
apparently do something completely
different depending on what prompt you
put in
it's quite interesting yes as you're
saying wouldn't it be so much better if
we could have a world model
we could have an understanding model but
how would you even train such a thing
if people knew how to train such a thing
i'm sure they would have done it
i think it would be a lot like training
a child so it's a lot of
hard work right because you have to
associate
linguistic expressions with things that
are not linguistic
but sensory experiences even that is not
enough like
it it has to have an understanding of
those sensory experiences
and that requires what yeshua banjo
calls
a system 2 architecture
so it is every major you know cognitive
function that we perform to be able to
learn
something like that i think it's that's
of course a pinnacle of intelligence
it's it's not easy for us it's probably
not going to be easy for
it's not going to be one machine
learning model simple machine learning
mileage but
scaled up but it will be a full-blown
cognitive architectural source i think
and that's my guess
at the beginning uh of the episode you
said
it very briefly that the systems you're
working on
have the ability to become human level
intelligent
do you maybe in the context of this did
you might want to elaborate of how this
since we determined probably gpt3 can't
go human-level intelligence how might
such a system
go human-level intelligence i i
identified in my work
a few principles of general intelligence
i tried to focus on those and try to put
them in my
architecture the architecture is called
omega
is it was published in aji twin end and
two years
before that in an ichikai workshop on
generality
there was a workshop on architectures
for generality and
so i think uh a lot of darpa people were
there too
i got some really hostile reviews uh
it's it's
still like you know this uh nurse at
high school
debating each other what i try to do is
make the system as complete and as
diverse as possible
and by complete i mean uh completeness
in the sense of turing completeness
universal induction so i think the right
system has to have built on universal
induction it has to be able to
search the whole space of programs and
not just
not just a narrow class of deep learning
architectures
that's one thing and second thing is it
has a very good memory
and then it tends to meet some
principles of generality which are which
i
identified as domain independence data
type
independence and task independence and
to me
all of that i use the principle of
diversity from solomonov's research
which he interpreted as well we have all
these machine learning models
like support vector machines and linear
regression
et cetera why don't we put these all
together
in a modular system but we have all
these diverse
methods that we can combine
arbitrarily and then this system would
be very powerful
so that that was his approach in his
alpha architecture i tried to extend
that by
by putting many
programming languages into the same
system
as well as deep learning architectures
so my system can generate well
in theory the entire gamut of
deep learning architectures as well as
solutions in a number of
programming languages and it can reuse
a lot of machine learning primitives
like decision trees too
so it's built on this principle of
diversity
but also universality and that's how i
try to
manage the generality aspect and this
kind of system
can get very intelligent provided that
its memory
works well so if you can't accumulate
can i just say that this it sounds very
ambitious and very cool
and i love the fact that you you know
you say you extend the alpha system
and instead of doing the beta system you
you say like now this is the
this is the last system that we'll need
like
this is like iteration done this is
omega
just like i i love this this is this is
perfect sorry tim oh yeah that's fine
that was a great great it's a funny uh
thing right i had a look at the
architecture and it was it was very
intriguing to me because it almost
looked like you had been
you'd gone shopping i'm gonna have the
decision trees and i'm gonna have the
bayesian stuff i'm gonna have the deep
learning models i'm gonna have a
little bit of symbolic as well i'm going
to put a bit of salt and pepper on the
top
and okay intuitively from a kind of
systems theory point of view
just because you've got all of the
components in there doesn't necessarily
mean they gel well together
this is legitimate criticism tim i know
i i welcome it
it's also what the panel panel manager
at
agi conference said like is is very
unifying principle in this architecture
because i can't see one but
that is that is the idea that all aj
architectures
have to be super reductionist and they
have to use
this basic component convolution neural
networks and it has to be everywhere
whereas my architecture looks very
heterogeneous and as you say
it looks like you've gone to shopping at
the machine learning mall
yeah it's more closely like it's more
closely like something like the brain
right because you have all
different kinds of stuff and even
throughout your personal
life if you just think of the methods
you use to solve problems sometimes you
you have a metaphor for solving a
problem oh never
look a a horse that you got as a
gift
i don't know if you have that in english
never look a gift horse in the mouth
yeah something like this and other times
you really think logically and so on so
it makes a lot of sense to have
a heterogeneous system but how
specialized is the brain
i'm not a neuroscientist so this is when
you know i'm asking here but
i'm sure keith made the comment the
other week that a lot of it is
kind of specialized what i recall saying
was that the brain
contains many specialized structures of
neurons
so it does have all these kind of
specialized components and some people
come along and say
ah yeah but you know your visual cortex
can take over for hearing if this part
of the
brain is damaged and i made the point
that can happen but it does so with
artifacts
so it's just wrong to assume that they
fully connected neural network with no
structure whatsoever
and this sort of dumbed-down neuron
compared to what actual neurons are
is necessarily capable of everything the
brain can do because the brain is you
know structured quite differently
right first of all the brain does have
heterogeneity in computation right it
has many kinds of types of neurons
we have let's try most of it i think but
is we're finding about
more and more about the different types
of varieties of
computation in the brain such as
indentative structures and so forth
so it it's actually reasonable to try to
extend our existing model classes
to be more universal with more
operations more primitives which i
really like by the way like it's i i
just want to put
everything in it you know so
kitchen sink omega right right so
so uh let me try to explain the
reasoning
behind it a little i said i i i hope you
like that
marvin minsky talked about this first he
had this chart
where he showed that for different types
of problems
different representations are effective
safe for some types of problems neural
networks are
and those problems are problems where
you have
a lot of inputs and where each input
has like a small influence on the output
right
so that's where you need a neural
network but
and then you have these very symbolic
problems
where if you represent within the
algorithm it would
be more suitable or in some cases
it's better to represent the thing as a
linear algebra system
so i try to capture this diversity of
mathematical representations in ai
and that forms the basis of my nearest
symbolic
ai unification architecture because i
want to have the best of
symbolic representations and i want to
be able to use the
right representations for the problems
where it's effective
and i want to do the same thing also for
neural networks
so in some problems neural networks
aren't the best choice and some problems
symbolic representations aren't say my
first work was
trying to do universal induction with
scheme
language okay but scheme is it's not
a great fit for representing
probabilistic
solutions or images or things like that
so obviously obviously you need neural
networks to do
images nowadays right so that's one
thing
so it's obvious that you can't just use
scheme
uh solomon of used a polish prefix
language
called azet in his design and that was
probably a darpa project
i think is i think part of it was
classified and
he published part of it and
right kate knows something about it so i
was doing scheme
but scheme is limited and it's hard to
use a search algorithm to find very long
scheme programs but
say to solve a typical machine learning
problem
you need a longish program so it's it's
like
infeasible for many problems so you
can't really use it but
what you can use it for what you can use
skin for is
is a glue language right so that's what
i use it for
in this architecture i use it as a glue
language
just like imax lisp so i use it to
expose
system functions itself i use it to
combine primitives and
stuff like that and i use other
languages for instance i use something
called metanet
which is a generalized or general
artificial neural network model
so i try to you know incorporate some
more stuff or primitives
more some more complex topologies and
stuff from
neuroscience but still including the
whole
convolutional neural networks and layers
thing this system can use
metanet to generate pretty much any deep
learning model
and then i have a probabilistic logic
reference machine which the system can
well
conceivably use for common sense
reasoning stuff like
if i hit this glass it might fall down
and
the water might spill so that sort of
reasoning required probabilistic logic
probabilistic logic is what open cog is
built on the whiteboard
architecture the atom space of opencog
is
a kind of probabilistic logic so as
gopher researchers think you absolutely
need probabilistic logic and so i
thought i
i had to include some kind of stochastic
logic in the system
and i have matrix computers so that's
good for working on scientific
problems say like controlling a fusion
reactor
or something like that or flying a
rocket
and bayesian nets again that's for
representing
some data science problems where we have
this complex uh
in interdependencies and a synchronous
computer
for dealing with event-like situations
like where we have say
synchronous events and and the system is
to represent them if you haven't
hey thank you it's i think i didn't get
to that yet but i think it is
it's right to percentage for a lot of
things like interactive computing sort
of things
which were already using uh graphical
user interface com
design but as i even see many examples
of within
aia and a picture language of tannenbaum
and that's from the darpa ppl project
it's i thought you need uh you need a
language to represent images
but you know it's a you can extend you
can add more languages
to the stack and uh it would become more
expressive so can this
can this get to human level it depends
on the um
efficiency of the ai kernel i use and by
the ai kernel
a general purpose machine learning
system that has
integrated long-term memory it's
interesting how you're describing it i
know
hoffman described intelligence as a kind
of operating system but i'm not
suggesting i'm not equating you with
that but it does sound in a way that
you're describing a kind of operating
system of lots of specialized
programs for doing different aspects of
intelligence i want to move the
conversation onto the simulation yeah
yes
before we go so just to frame this up so
nick i'm just going to try and do this
off of my head but nick bostrom had this
argument from simulation
it went something along the lines of if
we develop
the technical sophistication or any if
any human civilization out there
developed a level of technical
sophistication that allowed them to
build a computer simulation then
there must be many more computer
simulations than there are actual
civilizations therefore we probably live
in a computer simulation something like
that it's because the
simulation because you could run many
simulation and also because
within the simulation civilizations
would start that again
would start simulations that's good
exactly just to frame up your views on
this so
you've said that ancestor simulations
are silly you say that and the
simulation tech is implausible that this
stuff
requires evidence to be taken seriously
you've even said that it's a form of new
age creationism
you've said that we should be
open-minded is not a good argument for
taking this seriously
and it's it's interesting that it's a
bad argument for not taking intelligent
design
seriously which i think you've got quite
a good point on there and you also said
that
you're not convinced that we would be
able to detect a simulation if we were
in it
so yeah your thoughts on that i think we
might be able to detect a simulation if
it's real
but are we in a simulation that's the
question so
my counter argument is actually exactly
the
older counter arguments intelligent
design advanced by algorithmic
information theory specialists it is the
same kind of argument i said that
you say this but you need evidence so
the answer is
as carl sagan would have said
extraordinary claims require
extraordinary evidence and
in this case we definitely don't have
that kind of evidence
but it's not a claim it's philosophy
right yeah but the right
kind of right right kind of plaza
prediction philosophical or not is
induction
if induction theory didn't work then
machine learning wouldn't work at all
but you were talking about ethical
frameworks as well and presumably
there's no
induction there either right so maybe
you can take us maybe you can take us
through the
there's few steps to the argument right
there is step one
simulations of the universe are possible
step two
if they are then they
will be done step three if they
will be done then there are many more
simulated universes
because you can do many and within the
simulation there can be new simulations
then there can be there are many more
simulated universes
than real universes so it's it's like
just three steps
where in the three steps do you see the
the criticism
here uh first yeah you can't make
simulations of anything
right so we actually have simulations of
string theory and
big bang using those simulations
and they work by the way but they're
not big enough for them to evolve
you know world and intelligent life in
it this that's going to take a long time
so that's not how it works right it's
it's not like there aren't any resource
constraints if you ignore
all resource constraints all things we
know
about quantum computing physics
computer science the information theory
if you
ignore all this scientific knowledge
then maybe you can agree with the
simulation argument
but on the right hold on everyone
everyone at once because i wanna i wanna
pose
a thought experiment so i wanna pose a
thought experiment because i think we'll
probably all
actually agree on this which is that i
mean
all right you're an agi researcher right
so
i assume you're optimistic that at some
point in the human future
we will be able to create agis would you
agree with that
okay so at some point at some point in
the human future whenever it is
500 years from now 2000 years from now
it doesn't matter
we ourselves like this civilization will
be able to create
simulations in which we embed
millions billions of agis and have them
experiencing a self-consistent world
that has physics and
and everything like that what would stop
the
agi erai from making the same argument
to conclude they're not
simulated right it's um
[Music]
i think um
first of all it would rule out the
ancestor simulation idea because it's
really silly
so i suppose you want to say that world
is a remarkably different than our own
right
i it you know it's can be different sure
it's different from our own
yeah let's start with that premise right
so has slightly different physics maybe
it's finite maybe they've figured out
that the universe is actually a four
sphere and whatever but they have
physical laws and they're examining all
that
right right so you know like if it's a
simulation
there would be evidence that it is
so they would figure it out really fast
that's what would happen
if you ask me so it's uh this this two
situations aren't comparable because
they live in a simulation and we're not
living so what evidence would they find
like what evidence would they find that
they're living in a simulation
that we are not they would find limits
and
artifacts so that's what they would find
but they would also find some something
really funny with their physics
well but how do you don't know we don't
know that we haven't found limits
like we don't actually know that the
universe is infinite that's my opinion
okay so that's my opinion but the thing
is the cosmos is like
radius is 90 billion year light years
right that's big
they would make it big to stop us from
bleeding over
see this is why he calls it young earth
creationism that is
because that is i have this is one of my
specialties
right here one of my unknown special
this is this gets into the exact
arguments that young earth creationists
exactly and they'll be like you'd find
evidence but then
couldn't we plant that evidence as
makers of the cosmos couldn't we
put that evidence there and couldn't we
make it appear big but couldn't we put
the light
already into place racing towards earth
and
things like this i agree with their eye
like you you can never
you can never no no no this is just
silliness this is silliness that has
nothing to do with the
with the the frame so let me first let
me first state this
so so hey we don't know that our
universe is infinite
even though i said last time tongue and
cheek quoting a science fiction series
that the only truth is that the universe
is infinite
we don't actually know that it could
just be a finite four sphere that's a
hundred times larger than the observable
universe and so
you're making a truman show argument
hold on so as soon as this universe
is finite we lose
that sort of ability to say that the
people in our simulation would
observe a limited universe whereas we
don't
okay the finiteness of the universe is
no argument for
any sort of solipsism it's just not the
evidence we're looking for
right so if we're talking about i mean
other evidence
we have to be more specific so we're
talking about we would see like
problems with rotational invariance
because there's some lattice
on the line and let me be clear like in
all the experiments that have been done
to date
where physicists have looked for
artifacts of
lattice structure they don't find it
however
that's making the assumption that the
type of computation that would be
simulating this universe
is the describable kind of you know
turing machines or whatever
all right so instead of a real computer
for example
let me get to the bottom of that it
could be the case that digital physics
is true
okay so the cosmos could be a discrete
computer right could be could be this
still would not imply that we're living
in a simulation
it has nothing to do with it what does
the simulation mean what i'm saying is i
asked you in that hypothetical scenario
where i've
i'm simulating a bunch of agis what
artifacts would they observe
that would lead them to conclude they
are simulated and the example you gave
was limits and artifacts
and now you just said that they would
they would find okay
are you familiar with the red pill blue
pill
software if you're running in a virtual
machine are you familiar with that
one second which is you said they would
find limits and artifacts
and yeah yeah yeah they're a simulation
you said that now you just said
if we find limits and artifacts in our
universe
it does not imply we're living in the
simulation
no that's not the artifact we're looking
for is a
if you find that this is physics is
digital this doesn't mean it's a
simulation
okay so what artifacts with the
simulated people what artifacts would
the simulated people find
that lead them to conclude that they are
in a simulation
right so uh i think they would find that
their quantum quantum states are
somehow not quite correct and there are
some other forces
inaccessible to them but they can see
the
effects of those forces by dark matter
and dark energy right but
you see uh this is um distance to the uh
over speculative part to explain dark
energy or dark matter
we don't hypothesize a creator that's
like the
last explanation no no see this is where
this is where you and yannick go down
the
irrelevant area like the fact that
creationists
like like the simulation argument
doesn't mean that the simulation
argument isn't scientific
right right or can't be evaluated like i
don't i don't care what creationists
are up to right right right so um it's
not scientific
okay the simulation argument depends on
the blend
indifference principle right our are
i think we're down a fine path here so
you're saying again
i'm imagining this we're going to
simulate in the future some universe
with agis we all agree that's possible
okay and we're asking what artifact and
you're saying they're going to find that
quantum mechanics
behaves in a weird way you weren't
specific about what weird way
or that they would find the influence of
forces that they don't have access to
i brought up the dark matter dark energy
and your response was to talk about
creationists
that's not an argument it's they would
probably find more severe artifacts and
limits they would find that the
information they get from astrophysics
is fake
okay or they would find that it would be
interesting to be more specific about
the actual artifacts because then we
could go look for them
scientifically like other people have
proposed actually
if we look at cosmic rays they may have
some anisotropy
and in their velocities and things like
that so my point is that this is
actually
open to scientific investigation i don't
care what case so
this is open the scientific
investigation right those are
different kinds of questions let me
recap what the simulation
yannick summarize it perfectly but first
of all the simulation argument
in both chalmers and bostrom version
are strictly about ancestral simulations
right
so they they are about some post-human
programmer
god assimilating a previously existing
earth
with all this inhabitants so let me ask
you first
how is that possible it might be at the
evolutionary
it might not be explicitly programmed
right let me pivot a little bit because
we're in a local minimum here
i i want to ask an interesting question
which is you
have one no one one of the problems that
you have with these folks is that you
call it a new age religion
and you say that their ethics are wrong
because they the way they have these
utilitarian efforts they reduce
pleasure and pain in a reductionist way
and
i've got an interesting thought do you
think that they are
their ethics is faulty because they
believe in the simulation argument
because
if they were so nihilistic that they
believed that they were in a simulation
then it's almost like nothing matters
anymore and their
thinking on many other matters would be
tainted
of course um there's a relation they're
actually worried about simulation
shutdown risk
yes because these folks actually believe
that
they're in a simulation and that ai
doomsday is more
threatening to us than nuclear
armageddon and that's the only thing we
should be worried about so that kind of
damages their thinking processes
in other domains of course all forms of
creationism the values human life
and it's it's a way to deflect reality
if you're not
sophisticated enough psychologically to
accept reality as it is
then you develop these psychological
defense
mechanisms like afterlife and so the
mytho and afterlife or a heaven hell
or stuff like that they're all connected
but they're all
reality denial at a score and they
usually lead to very wrong ethical
systems so that's a big problem
and of course that's why because boston
is a creationist
who who presents himself as that's a big
claim
that's a big claim right so guys we're
totally leaving the realm of uh
science and logic here like i don't care
what people's
you know motivations what they learned
as a child i care about logic okay
so so can you please let me refute
robustrom and then maybe you can
maybe you have a version of it that came
to refute it but i don't think so
sure first of all it's very costly to
simulate
the world you need a quantum simulation
and in my essay i think you link to it
like uh the
simulation argument is invalid it's it's
a draft paper
that i will extend and try to publish
somewhere but i don't want to give
bostrom too much credit for it but
right so you see quantum computation is
very costly you you can't just
simulate it on a discrete computer you
need a quantum computer
first of all suppose that's actually
that's not true
that's right it is factually true so is
quantum computing doesn't doesn't change
the nature of what's computable one iota
okay so it it it does in the sense that
is
it requires exponential more states
for some problems not for others right
so so for
uh for simulating quantum mechanics
correctly
you need the exponential time okay so
that's not feasible
discrete computers whatsoever and some
physicists
after i wrote that essay published a
paper detailing that
but it is it's really kind of obvious
you know if you understand quantum
computing theory it's obvious
first of all you need quantum computers
and you need these gigantic quantum
computers
and excuse me what are you going to use
this for
ancestors simulations seriously that's a
mind that's
mind projection fallacy you're
projecting your kind of personal
preferences onto a future
civilization that you comprehend can you
please allow me to elaborate
let me explain how how damn stupid that
is
it's uh it's a fallacy
it's not a okay it's not a it's no
fallacy
okay it's called the refutation
but let me first explain it and then
you'll you're free to disagree
um now about the motivations of a future
civilization of posthumous it is bostrom
that makes the assumption that with what
with 50 percent probability they will be
making ancestor simulations so is
what kind of an assumption is that this
okay so that's also the mind projection
fallacy
yeah yeah let me continue please
let me demonstrate the absurdity of his
argument and then we can
try to find a better simulation argument
okay
what he's trying to do is basically
try to find a naturalistic thiagony
using the computer jargon right
he writes this in his paper okay i
didn't say that
he said it okay he's looking to
find some kind of you know matrix
science fiction
formulation of christianity that's what
he's trying to do there
he's talking about the hierarchy of
angels and such
those are all christianity but what is
the most christian
ideolo what is the central dogma
dogma of christianity can you tell me
what is the central dogma of christian
mythology
anyone yannick do you know what the
central dogma of christian
mythology is he'll tell you in any
second now
i mean that's important okay so i want
everyone to hear it and understand
what's so wrong
with that what is the central dogma
of christian mythology the uh
virgin birth death and resurrection of
jesus christ
yes god made man in his own image
right now that's the one sentence
that is not explicit written in boston's
paper but it means it
okay in the philosophical theological
language he
he explicit rights this is a
naturalistic theogon so that
are you are you saying that because the
ancestor simulation
means we are making people in our own
image
yeah god the post-human deity prog
programmer deity is making
us the simulated entities in his own
image
right right so that's why but this this
is your main
reason for saying he is a theist or
creationist
oh no the argument is explicit agnostic
is agnosticism in the first place but he
gives he gives
30 percent to the probability that
creationism is actually true and his
scientific is naturalistic so that's the
kind of
materialistic theogony can i jump in and
maybe i think we're also at time or so
maybe i can resolve it in a way that
make makes
everyone happy a little bit so
let's agree that maybe
if we can do such simulations
and we might be doing an ancestor
simulation
or something like keith suggests
we could feasibly forge a world
that beings in it could
live and that might very well be however
we have we have no indication no
scientific evidence that anyone can
point to except this
sort of if if you know if simulation
exists then there are many more
we have nothing in the physical world
that anyone
can point to and i agree with keith a
bit that we're not really sure what
we're looking for
like artifacts i don't know but
there's nothing that right now we can
point to and say
that thing that thing means we're in a
simulation right
as of now we have none of that what
makes me
to me personally the biggest argument
and i agree with arai a bit
right here is a let's say a
thermodynamical argument or a
computational argument
if you want if we were to do
what keith suggests and simulate a
universe
with let's say beings in it and let's
say
we can't simulate our own
because that would require let's say a
computation
like so big so all we could do
is simulate somehow a reduced universe
like with more maybe consistent maybe no
artifacts at
all right but i but you can but it it's
somehow
reduced in its complexity and that means
to me
that means the there can it can't go on
right the simulators can then simulate
again and simulate again indefinitely
because there is always this kind of
reduction and at the end
the simulation just consists of one bit
of going like boop boop
i don't disagree with any of that by the
way yeah okay so
maya has something like first up for a
computational limits argument is a very
strong argument against it
quantum computation is very costly and
to simulate physics accurately you'd
need a lot of quantum computation and
you can't just
waste the precious cosmic resources for
a silly video game
that's not something a super intelligent
entity will do okay
once humans are eliminated from the
equation nobody will give a
flying f about us so that's not gonna
happen
but a more important objection is of
course
the argument from induction right when
all
else are equal right we prefer the
simplest explanation not the most
complex schizophrenic explanation so
that's why we reject the
simulation argument and i gave a very
good figure for that and
how much is that so it's kind of cool
that's about this
what is the size of the human brain how
many bits
would you need to represent it like
petabytes according to induction theory
if it says like one petabyte i don't
know how much is that like a trillion
right say trillion bits right
so the difference in a priori
probabilities
of evolution versus this creationist
view
is 2 to the power of 1 trillion bits
that's a lot so that's why you know
see my energy prior refuse
boltzmann machines both both my minds
right like spontaneous
both membranes uh definitely
it's refused refusable both membranes
are impossible because of this
because of the energy prior the universe
will prefer
energy efficient processes and that's
why
they don't just spontaneously form
that applies to everything and that's
why if you don't have extra evidence
to offset that huge change in
a priori probability and that would be a
very extraordinary evidence
like some computer god appearing in the
sky and pointing at you or something it
would have to be something
it would have to be a miracle someone is
trying to validate
biblical biblical miracles
with this okay young earth creationism
for sure
because you need a small computer so
that means they're thinking look look
they're thinking in their singularity
mythology
that they this post-human data scan the
minds of
all the people who who were living in
2045
and scanned all of their brains and it's
retrieved that information
and then reconstruct them and this
simulation
and we are those people that are being
stimulated rather than real ones
that's a really bizarre kind of young
earth creationism it means that the
simulation exists probably like
100 years or so and that's it right
yeah so let me respond to a couple
things so first off people
oftentimes attach to various scientific
kind of theories or facts to promote
their particular
agendas or they rail against facts that
don't agree with their agendas like for
example here in the u.s
we have lots of people that are in
denial about human biodiversity because
they're afraid that racists will
use that to promote racism things like
that i have to say while i'm interested
in those kind of
sociological aspects of it they have
nothing to do with the kind of argument
that i'm trying to make here
and i don't care if bostrom goes to
church on sunday or not like i'm only
looking at
the logical basis of the arguments
themselves but but
you cannot discount human motivation at
all because both strom
and tegmark were funded by templeton
foundation
which is pretty much a religious
fundamentalist organization that's
trying to prevent the
disestablishment of the church right
that's fine i'm gonna i'm gonna address
their arguments like head on without
worrying about like their motivation for
having
made those segments in other words i'm
not interested in ad hominem basically i
don't care what their motivations are i
just want to address
i want to address their arguments it's
not at home in them they got paid
to do this yes
that's my definition i'm pretty sure
that's consistent with the accepted
definition of ad hoc they still want it
let me respond
a couple of things which is
hold on let me respond to a couple
things so first off
and again my thought experiment is in
our future
we're gonna definitely conduct
simulations and we're gonna
you know of the universe and whatever
right let's suppose i'm my future self
and i spin up a simulation and i put in
some parameters and i start simulating
a universe because i find it interesting
and i have the resources to do it and by
the way people waste resources all the
time like look at bitcoin
you know it's inferior in like every
single way transactionally
computationally whatever but we're
spending huge amounts of you know
about energy resources you know
calculating it
so i spin up my calculation okay now
listen
i spin up my calculation with parameters
and it may happen that on some planet
life evolves i may not have cared like
that may not have been the intention of
my
simulation i may have just been trying
to explore different type of cosmologies
and it happened to be a universe that
was favorable for life and so some agis
spin up in my
simulation like whatever you know i mean
i'm just i'm just observing things
but these resources are not infinite
that's a lot of assumption
i don't require infinite resources to
compute a finite universe
and and again we don't even know that
our universe is infinite
it certainly appears to be finite in the
past
exactly our universe is finite that
means computational resources are
valuable you can't waste
on some nonsense cosmic stimulation it's
it's impossible
right yeah but this is this is to
yannick's point is i'm not
my future self is not going to simulate
our universe i'm going to simulate
other universes that i have the
resources
to simulate and my point of making here
is that agis may crop up in that
universe
and they may have a zoom call just like
this one where one of them says like we
can't possibly be simulated because we
don't have the resources to simulate
ourselves
that's not logical but it goes against
it does
refute this argument that these
simulations can somehow
spawn themselves continuously right
because
has anyone made that argument i mean no
is that does bostrom say that there's an
infinite regress of simulations to get
the overwhelming probability
you sort of need that right
excuse me the 30 percent probability
perhaps you would like to read my uh
draft paper
why the simulation argument is invalid
after the talk
because i actually respond to that it's
a valid thought
what is plausible in the manner of
simulations there are two things when is
your scenario maybe some
post-human entity agi is using
simulations
and for scientific research so that's
plausible it's not
silly like ancestor assimilation which
is completely implausible
but so you want to research cosmology
and you're
making a huge cosmology simulation would
there be enough resources in it for a
lifetime well
i don't think so i don't think you can
do that by the way all right
because it would take approximately the
same time
as universe itself for that to hope for
so that's actually infeasible but
suppose it was somehow infeasible but by
some tricks i don't know why maybe it
was not evolution with something else
okay so that's also possible
does this uh relate to us like it
doesn't really because
all such in all such cases the
astrophysical measurements
would have to vary and we can only make
those
arguments sound convincing by way of
ignoring
the extreme discrepancy that would be
revealed via astrophysics
right so well i've asked you to be
specific about that and you haven't
given me any specifics but let me just
let me just say
so that's
excuse me the age of the universe that's
why we say young earthquake creationism
by the way
because obviously the age of the
universe would have to differ
so i've i've conducted many simulations
myself where the pace of time and i'm
talking about physical simulations where
the time
you know uh scale is very different in
my simulation than it is for me
like i said thousands of years you know
within
within kind of you know a minute or
whatever so you're just
assuming many more resources in older
universes than ours and etcetera so
these are just like
speculations that are not supported by
any evidence whatsoever
so they are not really things we should
consider
unless there is some reason to do that
you see these are bordering on
metaphysical speculations
so they are not they are not religious
okay so this is not religious
but it is just as irrelevant to our
scientific worldview hold on so
people thinking about simulator for that
matter just thinking about whether the
universe is computable
right whether the universe is a massive
lattice computation
or whatever people thinking about that
kind of thing come up with lots of
interesting
experiments that are let's look at
anisotropy and cosmic rays let's run the
halometer over and wherever it is
excuse me those are for detecting
whether digital physics is true that's
valid for science okay so that's the
second thing
let me point it out could something
about the idea simulations somehow carry
over to some scientific thought
and and i get given affirmative answer
in in my essay
and let me say that annually agree okay
so we'll find the
a common ground or agreement it might be
the case
that we're living in a computational
universe
just as edward franken or stephen
wolfram would say
in this case the idea of a virtual
machine
might be a good model of how a universe
is formed
so it could be that we live in this
bulk with a lot of brains
and each membrane could be a virtual
machine
with its own physical laws right
and these could be basically automata
that
evolves over this larger computational
bulk so this would be a very scientific
worldview because it doesn't
involve post-human gods creating
experiments in the lab or that's not
possible so
this is how it might be viewed in a more
scientific way
via the lens of evolution so
if we're saying it's evolution it's
scientific if we're thinking of
some gods making universes it's not
scientific
it's there's no evidence for that sorry
so here's the thing is
if once a universe and i'm just going to
use this generically once a universe
is a computation and finite
it's possible that it's a simulation
yeah but it's possible well so look i'm
not
look i'm totally with you on the you
know when somebody comes along and says
look i have an argument that has
three terms and therefore bayesian
analysis tells me that each one
is 33 percent or whatever excuse me
can you hold there that's not based in
analysis in base analysis you have
priors
i told you the right prior evolution
theories much more likely is how much
more likely 2 to the power of 1 trillion
times more likely i'm i'm with you i'm
with you on this okay
and that's the answer excuse me and
that's the answer to your further
variation
in the future if the agi were pondering
they have no evidence there in a
simulation okay
and they ask this question but they are
in a simulation they just don't know it
yet
all right and they're asking each other
none of their experiments
indicate simulation over
evolution okay so they believe they live
in an evolutionary universe
okay so with normal physical laws
because they're smart they figure it out
but they somehow arrived it's wrong
answer okay let's get all that
let's assume this might be the case it's
a very remote possibility by the way
because they're smarter than us but if
this might happen
maybe they got something wrong they have
their science
is basically wrong right
their physics is wrong so uh they have
the wrong
idea wrong fear they believe in the
wrong thing so it's the exact opposite
of this universe
so you get that right this is it's not
it's apples and oranges by the way
now it is listen to
listen to this and they're asking each
other just as you said oh
do we live in a simulation we have no
evidence what should we conclude
they should conclude that they are not
in a simulation
okay so that's the right answer are you
surprised
excuse me you should be surprised
because you thought
if they thought right if they thought in
the right scientific way
no i'm not surprised because i
understand the difference between
realism versus not realism versus you
know
etc and so i'm not surprised that they
take a philosophical view
that their universe quote unquote is
real right
that's right that's fine i'm not
surprised by that no no
it's because they don't have the
evidence for it it's natural that they
will
they will not change your thing and it
will indeed be the case according to
them
my argument will still be valid okay in
their investment so until they've
recovered evidence for their case
which might not be recoverable after all
right maybe they will never find out
maybe somehow they won't be able to
build the devices that will detect those
artifacts or
whatever maybe they are not buildable in
that universe maybe you program the
simulation so that those devices will
never be
built so that's possible but that
doesn't mean that
we should believe that we might be
living in a simulation at all there is
absolutely no relation
so we've shifted context here because
what i was pointing out and we have to
say
i understand but contexts have been
shifting theologies coming in
all kinds of stuff what i'm saying is
that certain arguments are not valid
arguments
against simulation one of those
arguments for example
we don't have enough resources to
simulate our own universe it's not an
argument that we're not in the
simulation
saying that i don't understand why
future beings would want to do
simulations is not an argument for us
not so i'm just pointing out there's a
lot of logical fallacies
on both sides of this argument i'm sorry
those are not logical falses at all they
are
very well as refutations well the
objections no no
fallacies like i can show you
symbolically some other time
oh there's just a straight up mind
projection
i'm sorry like jodel tried to prove god
with model logic
gentlemen i think this is an excellent
time
to draw this to a close anyway this has
been this has been one of the most
animating and interesting episodes we've
ever done i've
i've never seen culture so animated
before
because i feel like we've been doing
this um epsilon greedy thing
you know where 10 percent of the time we
explore a new part of the search
space and i think we finally lit culture
up so
this is not that he's not normally lit
up but today he was lit up like a
christmas tree
anyway um dr alai uskarov
it's been an absolute pleasure we hope
to have you back on thank you very much
indeed
a pleasure thank you thank you a
pleasure as well as i'm hoping to see
you guys again and
thanks for putting out such a great show
it's been so much fun
it's an absolute pleasure thank you
thanks so sorry
okay what i mean is that's essentially a
creationist view if you believe that
there's some magic
biogenesis that's creation is the main
way
you understand that i'm sure abiogenesis
just refers to the point at which
there was no life before this point and
there is now
it's nothing creationist about that i'm
not saying it didn't arise through
naturalistic processes
i'm just saying our understanding of the
naturalistic processes that could have
allowed
life to arise points of it being like an
excruciatingly low probability event but
it did happen
right no it doesn't uh so that's not
true we have it here of self
organization it means
it's probable yeah and that all falls
apart when you actually look at the
details
of this form of life of rna evolving
as it did in the early environment
okay so so you understand that this is a
general form
of an argument it's called the god of
the gaps
yeah this isn't i'm not arguing for a
god of the gaps what i'm saying is that
lo there's only for anything that
happens there's only three explanations
either
necessity it's a necessary consequence
of physical laws
or chance or intelligence
those are the only three possibilities
and i also take issue with people who
think that
intelligence is not something that's in
other words
science can very easily analyze whether
or not an event
was the result of necessity chance or
an intelligent agent like we do that all
the time in forensics
if a house burns down and i'm analyzing
whether or not it happened by accident
or by the volition of some intelligence
that's science so it's
one thing i find absurd is that people
out there who say no no
intelligence isn't analyzing whether an
agent did it is not scientific that's bs
right so we have a scientific
explanation of biogenesis that's not
true
okay there is no scientific
we do it we do it's called
self-organization and it's called
yeah it does it's called the free energy
principle and it explains
life and consciousness yeah i read your
chapter that you sent on that and it was
all very
hand wavy no actual theories no actual
equations
you should you should have phone no
actually you should follow the latest
references
explain you have to explain this form of
life
not arbitrary forms of life our form of
life
no it's arising on the earth's
environment as we scientifically create
a free energy principle applies to all
forms of life not just ours
you have to explain this form of life
arising on
this earth as it was four billion years
ago
yeah not just some general principle
that says it could happen
okay so you're looking for an experiment
for which we have no data about right no
no no no we know a lot about early we
know a lot about early earth
what environments no no we don't have
samples so we don't have
the evidence like you're asking for
something that's basically impossible
we can't we have samples we have rocks
that were there we understand we don't
have samples of the life then
so we don't have the uh we don't have
archaeological evidence to reconstruct
it fully
you're asking for something that's
basically impossible but we can create
look we can create
we can simulate biogenesis on computers
we can't do that that's what we can do
right right we can prove
the free energy principle and if you
follow the references you will see
that it's very sophisticated
mathematical theory
that paper is philosophical it's the
only paper that tries to explain it in a
philosophical veneer by the way like
it's
it's unique in that regard every other
paper on free
energy principle just follows the
mathematical formalism and it's
it becomes grotesquely incomprehensible
but so it's very difficult theory by the
way but
it does explain the very thing that
you're asking
and if karl friesen told this to me in
person
free energy principle explains
biogenesis
yeah it doesn't it does i mean you can
keep asserting that
but until you prove that it explains the
the abiogenesis of our form of life on
earth's
early environment it doesn't prove
anything relevant to this earth
okay you're not right it's if it takes
itself or
if it explains self-organization then we
can generalize so we can
we can have good confidence that it also
explains that but
can it explain self-reproducing
chemicals
can it explain the evolution of cellular
organelles or mitochondria or cellular
membrane etc those are all separate
experiments
that will probably be carried out in the
future but so far what we have recovered
is
and that's very curious by the way is
that does explain the behavior of
neurons
so that's very remarkable it's
something that was not explained by any
general theory before that
and we believe it's also explains the
inner working of the cell just as well
but those are very difficult experiments
what i proposed the first time to do
that was exactly the assimilation
so we will be making some gpu
experiments to confirm that it's
possible but it would
by the way it would have to be quantum
mechanical so it's very difficult
i don't know if we have enough compute
power to do that but
it's conceivable that we'll be able to
show some kind of non-trivial evolution
of chemical materials so that's not
genetic evolution right
it's it's chemical evolution
and that's what the theory is about as
free energies gibbs energy it applies to
all chemical
substances yeah so when you when when
you get those results
you know but but but we don't need to
that's the beauty of here
do you think einstein needed to
prove every application of general
relativity to
present any conclusions about it yeah so
we're absent any specifics here so i
can't really
respond in a meaningful you can't
meaningful way
i i'm telling you like you should follow
the references it's a lot deeper than
you assume
and the mathematical formalism is of
course very sound like you take this
ah yeah i'm so sorry guys i've got to go
but okay all right
it's been an absolute pleasure talking
with you yeah yeah leave that browser
window open until it uploads
uh the wave okay so so see i'm sorry if
i've offended you in any way keith
like i don't i just try to do my i don't
know keith loves it
all right i am look um what was this i
saw the other day
somebody has this great quote which is
they say that nerds are people who think
the purpose of communication is to
submit their ideas for peer review
whereas whereas everyone else thinks the
purpose of communication
is to negotiate agreement
i love this like i mean you know this is
uh this is why i'm on this
you know podcast frankly so no i have to
say it's i have to say two things one i
have to say
this conversation today was completely
not what
i was expecting it exceeded my
expectations in every way i had a lot of
fun
talking to you i think you and i would
actually get along great in real life we
like to hang out at conferences and have
a lot of fun so
i hope we can meet someday when corona
is under control
and i don't know i think we should do
more of these conversations it was
just fun you know i really appreciate
your you know like you
you took a really technical approach to
you to everything that we talked about
so that's very cool
and so you obviously have a very good
understanding of machine learning theory
so
it's the these are some other uh
subjects like it's
there are these are subjects of some
ongoing debates so naturally there are
there are bound to be disagreements i
expect that it's it's all right so
yeah i'm really looking forward to this
one reference you gave me here
i'm so looking forward to reading this
probably today which was the
the ultimate intelligence part one
physical completeness
and the objectivity of induction that's
going to be fun
i hope you like it great to talk to you
thank you yeah talk to you later
thanks a lot and see you around tim and
keith anyway i really hope you enjoy the
show today
we've had so much fun making it remember
to like
comment and subscribe we love reading
your comments
and we'll see you back next week