#029 GPT-3, Prompt Engineering, Trading, AI Alignment, Intelligence

welcome back to the machine learning
street talk youtube channel
today yannick and i in particular
changed our mind about gbt3
a little bit v-drag posted a very
interesting comment on the kanalihi
video
a couple of episodes ago giving some
examples of some of the really quite
impressive things that gpt3 can do
i didn't know about this gpt3 database
example that you brought up
at the beginning of wallet summit that
is crazy that is legit
crazy i was going through all of his
things one by one kind
of discarding them saying yeah that's
rubbish it's still memorization
yeah that's rubbish it's still
memorization and at some point i had to
stop myself and say
whoa wait a minute this is unreal
we've described a computer program using
natural language
and we can interrogate it using natural
language
this could potentially be a new
revolution in software development
if it is memorization it's such a level
of sophistication and nuance
that it appears as if it isn't
memorization
yeah or it's if it is memorization it
it's like
memorizing the highest level concepts or
something like this
and and that i think that's something
that connor
from last episode would agree with
intelligence and lot of sense is about
learning a generating function if we put
it in a box
in a virtual box with physics with a
universe that's completely unlike our
own
then of course it won't learn like
physics won't learn a good quantum
theory
because it doesn't have to but there's
no reason it couldn't
we believe that this can lead to a new
software engineering persona
someone who is skilled in the craft of
designing prompts
that make a model like gpt3 do something
that we want it to do
be this reasoning or intelligence or
whatnot i firmly believe that
a profession in the near future will be
prompt engineer
if as soon as something like gpt4 comes
out like that is just
even like an order of magnitude better
you can start applying this
in business context even now and then
like
prompt engineering will become a
profession i'm not saying you can
transition from being a poet
to being a prompt engineer like most
likely you're going to be
a software engineer but then you're
going to transition to prompt
engineering which is still a form of
programming
software 1.0 is writing code software
2.0 is trading models software 3.0 is
querying large models
and in a way this seems weird to people
but it's actually not that weird because
that's how humans work
90 of software engineering in my
experience is trying to figure out what
the your client wants
prompt engineering is in many ways
taking a thing
that can do many things and trying to
tell it exactly what you want it to do
in the olden days we used to have the
concept of a sequel injection attack
people could manipulate the input of
your program to
read something from your database for
example just imagine the kind of attacks
that could be possible if you were
directly interrogating
a gpt style model in a way this has been
proven out with the fitness ai knowledge
based beta
it's a wonderful tool you can
interrogate a gpt3 model
but it's been filtered so you're only
allowed to ask it questions
that are fitness related but clearly if
you manipulate the input
in a clever way you can make it do
arbitrary tasks
if you just formulate the query to look
like it's fitness related
what excites me most about gbt3 is its
ability to generate
code code is robust a sorting algorithm
can generate to any permutation of
numbers
we always have this question with deep
learning code that it doesn't have that
broad generalization that real code does
and the wonderful thing about something
like gpt3 is it can interpolate between
language and programming code
you can ask it programming questions and
language and you can ask it to generate
programming code with language and
anything in between
you might say though this is incredibly
dangerous who knows what kind of code
could be generated by gpt3
maybe you could even get gbt3 to
generate the static code analysis
the security checking you could have a
whole hierarchy of modules which are
created using natural language on gpt3
it's pretty interesting so when we have
data scientists and software engineers
putting things into production
we have loads of static code analysis so
now we're going to have another gpt free
model that says
i want you to generate some static code
analysis code to check that this code is
not malicious we can just build up a
taxonomy of these gpt-3
modules why not and we're never writing
traditional code anymore
code is now the artifact right now
you could probably make gpt 3
do a lot of business things by good
prompt engineering
i'm very convinced of this not
everything like you couldn't go around
solve every problem but you could solve
a lot of problems
that if only you engineered a good
enough prompt
you could probably make a reasonable
business case
we spoke about our collective
frustration in academia when we were
doing our phds
and keith told a fascinating story of
how he pivoted from academia
and became a wall street quant i've
become a lot
disenfranchised with academia and that's
why i became disillusioned and so
i actually went to the other extreme i
said what if i have to deal with
politics and money i'm gonna go into a
context where that's just
explicitly the goal like the only thing
that matters is just
making money however you do it in such a
way that you don't get
sent to jail are markets fundamentally
predictable
so for a lot of time at least when i was
running my own portfolio i was doing
what's called high frequency trading or
low latency
trading the only reason we can't exploit
the market
is because we have intelligent actors
exploiting it
against us do you wear a quant on wall
street
i've always been super interested in
this as and how much of it is smoke and
mirrors
when you go into the airport you have
this concept of security theater
where everyone makes a big song and
dance about security and i've always
felt this way about quants
that they make a big song and dance
about all this technical analysis and
all these sophisticated algorithms
and actually that's just a smoke screen
so
for a lot of time at least when i was
running my own portfolio i was doing
what's called high frequency trading or
low latency
trading you have a very high probability
of making money
almost all the time in small amounts and
then once in a while
you have a tail loss that takes out all
that you've ever made
you sell deep out of the money options
you're selling insurance it's like most
of the time you're gonna make money
and then once in a while some natural
disaster hits
and you go broke you lose everything is
predicting the stock market
or doing technical analysis on the stock
market
or even using social media as signals
for trading
is that stuff completely crazy or not
are stock markets fundamentally
predictable
it's a reflexive chaotic system which
changes as you
interact with it this is a real problem
that we have in reinforcement learning
at the moment
you are not interacting with a live
real-world system
so to what extent can you trust your
offline policy
will work in the real world and how
predictable are markets
we talk about the market efficiency
hypothesis as we were speaking about gpt
3 and intelligence in today's episode i
also got connolly he from a couple of
episodes ago
to come in and give us his take let's
talk about wire heading let's talk about
ais like
i say as stupid as it is that whole
experience actually totally changed how
i think about alignment
because i real because i was like oh
 if
ai's just give us what we want this is
what they're gonna give us this is
what's gonna happen
i wish i was an agent that doesn't
capture its own reward function
connor gives us his take on the economy
free markets
and capitalism the economy is a
misaligned ai
if if markets give us good things i'm
pro-market if markets don't give us what
we want i'm
anti-market i think free markets have a
good shot as being the best idea humans
have ever had
and there are just very well-known
failure modes of free markets
connor also gives us an economics
thought
about a dragon imagine a society of
immortal
you know wonderful happy humans living
in like some primitive little society
and they're all super happy no one ever
gets sick no one ever dies everything's
wonderful
one day a dragon comes and the dragon
says you have to pay me gold or i'll
start killing you
there's a surprising link between
economic utilitarianism and ai alignment
research
how do we even define what good looks
like these problems are incredibly
difficult
and no one knows what to do about it
utilitarianism is not
unified there's different strains of
utilitarianism that have different
definitions of what utility of what
happiness actually is and
which should be maximized is the
relative poverty
important is the absolute probably
important is there some kind of
thresholding
maybe that's implying that we should be
exploring the human condition as much as
possible
and getting stuck in these local
minimums or wireheading
where our senses get deranged
surely that's the worst possible thing
for human flourishing
and we come to some primitive village
somewhere in whatever
and they say hey hello we offer you
immortality and internal happiness and
they say no
should we accept that or should we say
no we know better than you this is a
really difficult problem
yes it is it's a very hard problem does
randomness
even exist how do you know if something
is random the question is if you have an
algorithm
let's say you have a cellular automata
yeah like this random
black and white cells that produce some
kind of pattern and we happen to find
one super bizarre weird cellular
automata
that just happens to output only correct
truths about the universe all the time
not because it has any connection to the
universe it never perceives the universe
it just
happens to have that mathematical
structure is it knowledge
is this intelligent how would you know
that something is random
and not just the output of some very
complex
system like where is the difference
between
a large number and a random large number
so you can make algorithms that output
numbers
that you can't prove are not random
in a certain amount of time one of the
key tenets in ai alignment research
is this notion of super intelligence
connor believes that there is
effectively no environmental rate
limiting step
in intelligence and this is the reason
why he believes that we are
in imminent danger of the singularity
the intelligence explosion
i think this is what you meant when you
said that as the abstractions become
increasingly weird
we don't really understand what they're
doing anymore but it seems to surpass
human intelligence so people are smarter
than me that i trust
are smarter than me that make good
decisions are telling me dude
if you understand conformal field theory
everything makes sense i'm like
all right dude and that's within human
levels and that is just so wild to me
that there are things where you can't
even explain to me what it is about what
the goal
is or something until i spent years and
assuming i have a high enough
intelligence you'd understand what the
hell these higher group theory is or
whatever
to even explain to me what this might
maybe potentially mean
i believe that the difference between
the dumbest human and the smartest human
on the scale of all possible
intelligences is very small
we were talking before about the
dichotomy of an embodied intelligence
versus something which is far more
nebulous and externalized but you think
it is possible for a single model like
gbt3 given
some incredibly esoteric abstractions to
do things which are just beyond our
wildest imagination
imagine if you could just read all
papers and do all experiments yourself
and integrate
all of their consequences all the time
which is form deductions based on all of
these information
that's a lower limit on what a truly
powerful intelligence should be able to
do
we have already passed the limit of what
a human can actually accomplish with
their own intelligence the way of
computers
you can't do a radio astronomy on
pencil and paper it's just not possible
that doesn't mean the information isn't
there it just means it's not accessible
to us
we are the dumbest possible species that
could develop industrial technology
because we're the first ones to evolve
if we had an agent that is magnetic
order more efficient than us or has
magnitudes of order more compute than us
we should expect it
to make magnitudes of order faster
scientific development i really hope you
enjoy the episode today we had a lot of
fun making it
remember to like comment and subscribe
we love reading your comments
and we'll see you back next week
somebody might scoop your thesis before
you're done writing it
oh i don't mind i don't care like i just
want to hand this in and be dumb
i know i remember what that was like
that was the hardest thing i ever did
was finishing that
god i've become a lot disenfranchised
with academia
that happened to me towards the end but
then when i went to postdoc at ibm that
was the final nail in the coffin because
i just i got tired of the
 and the hype and the politics
and the money and
i realized that if there are these
personality attributes of let's say
intelligence hard work and ethics
you can be not so bright and okay
because as an academic because you just
don't know what you're doing is wrong so
you're happily
happy to publish it and if it turns out
to be wrong oh you can be
intelligent and really hard working
and successful because then you've got
the work ethic to go through all the
rings that you have to not offend the
wrong people to publish to spend
inordinate amounts of time
writing about all the in a nice
way that doesn't piss off anybody
to explain why what you're doing is
different right huge amounts of work
you can be intelligent and lazy or
normal work ethic
and unethical and then you can publish
because
you don't mind if what you publish is
just flat out wrong even if the person
down the hall
has showed you that it was wrong because
you'll publish it anyway
and by the time somebody tries to repeat
it five years from now and waste
250 000 of funding on trying to do that
don't go who cares that's in the past
you've got tenure you're protected
whatever
but if you're intelligent and just an
average worker
and too ethical that's a really tough
spot to be in
as an academic right yeah i must admit i
became disenfranchised as well
i had nightmares about my phd sometimes
i still have a recurring nightmare
which i presumably used to have all the
time that i haven't finished my phd yet
it went on for about five years and i
thought i would never finish it i tried
to quit about five times
it was the most isolating depressive
period of my entire life
in a way it was really good as well i
clearly discovered myself and became
good at lots of other things for
procrastination
but that's why i really like
re-embracing the subject now in this new
mode
through yannick and through this kind of
collective
renewed excitement that we've had i wish
that i could have learned about the
subject in this way when i was at
university
once you have to go deeper and get
engaged with the peer review process
and and are fighting for funding and all
that kind of stuff that's where the
politics
hits the road and i don't think you can
escape politics in any form so there is
academia you have this giant politics
but then also in like big companies
like it's almost nothing like it's all
politics and gaming the kind of key
metrics
uh that give you promotions that give
you recognition and so on it's very
little actual contribution to the bottom
line of the company
so i totally agree with you so i was
naive and i believed in the
the ivory tower and that's why i became
disillusioned and so
i actually went to the other extreme i
said what if i have to deal with
politics and money i'm going to go into
a context where that's just explicitly
the goal
like the only thing that matters is just
making money
however you do it in such a way that you
don't get
sent to jail so i went to wall street
like that's why i ended up
defecting to the complete dark side and
i enjoyed that for eight years because
there were there was no politics
at all it was just did you make money or
not
and that's it and if people if you made
money
you could walk around half naked like
dumping coffee on everybody's desk
and still not get fired it was just he
makes money can't do anything
sorry but after a time like i started to
realize how
it was rewarding financially but very
shallow like just a very all i'm doing
is moving
money around and my kids ask me what i
did for work one day at dinner and i
said
i take other people's money and i move
it around
and every time i move it i take a little
bit and they just thought that was like
the most
awesome job description ever like
oh my god that's just awesome but then
that's
that got me thinking about how man like
i'm just i'm not doing
anything do you wear a quant on on wall
street
i've always been super interested in
this as in how much of it is smoke and
mirrors
when you go into the airport you have
this concept of security theater
where everyone makes a big song and
dance about security and i've always
felt this way about quants
that they make a big song and dance
about all this technical analysis and
all these sophisticated algorithms
and actually that's just a smoke screen
just because of the market dynamics and
the control and influence they have on
the market
surely that's the important thing not
the technical analysis
yeah so i'll answer this from two
directions there's some truth from that
so number one
the the truth is that so i'm sure you're
familiar with the concept of
tail risk probably if if not in case
this ends up on the internet i'll
explain it which is just that
imagine i come to you and i say dude i
discovered this
awesome strategy like it makes money
every day for the past three years
like every day wow amazing right
actually probably what's happening there
is you've distorted the probability
distribution to where like you have a
very high
probability of of making money almost
all the time in small amounts
and then once in a while you have a tail
loss
that takes out all that you've ever made
because it's very easy to construct
those strategies like i'll give you an
example is just
you sell deep out of the money options
you're selling insurance it's like most
of the time you're going to make money
and then once in a while some natural
disaster hits
and you go broke you lose everything and
in the industry so let's call those
tail risk strategies they take on tail
risk as opposed to
you know selling tail risk if they sell
the tail risk and therefore they're
exposed to it
so what happens there is that every year
i get my cut right
like i made a profit for the fund this
year and you gave me a piece of that and
next year you're going to give me some
and next year you're going to give me
some and then when it blows up and i
lose everything that's ever been made
and then some there's no clawbacks right
you can't come and force me to sell my
home and
and dredge up all the money i've spent
over the years and give it back to you
so the people who pay are the investors
but the problem is that people
psychologically are very
short-term focused so they like those
strategies
they like to put money in things that
have some hidden
tail risk that nobody can calculate if
you came to somebody with that option
strategy everybody knows that one
like we figured that one out so you're
gonna be like oh yeah all you're doing
is selling options so what happens is
either consciously or subconsciously
through sort of darwinian
natural selection almost the market
evolves to create these strategies that
have that form of risk that nobody can
calculate
can i try a couple of things with you
because i've been exposed to many data
scientists in many walks of life and i'm
gonna
deliberately obfuscate this so you can't
tell where this came from but i want to
give you a few examples of things that
data scientists have
proposed as being a good idea and a lot
of it is market dynamics so one
is pricing so they would say why don't
we back
test some pricing strategy on a whole
bunch of prior data
and if it worked on the back test then
we think it's going to work going
forwards another one might be optimal
storage strategy so i'm going to
run some stochastic model on a storage
strategy based on historical information
and if that works then i think it's
going to work going forwards
or another thing i've heard is why don't
we ask lots of people for
their opinions on whether certain events
are going to happen in the marketplace
and if those opinions have historically
paid off then we think we should be
asking
their advice going forward did you see
the pattern here
yeah we used to have to put this
disclaimer at the bottom of every single
kind of
investor report or whatever we sent out
that said something like
historical success is no indicator of
future success or something like that
the market is a dynamic system and
yannick talks about all this
talks about this all the time in terms
of exploring the state space and so on
yeah yeah
and then it's going to react to you but
yeah so you can only explore the state
space directly by interacting with the
market
if you try some novel strategy it's
unexplored by definition
yeah that so the market's going to react
to you like i'll tell you a funny story
so
for a lot of time at least when i was
running my own portfolio i was doing
what's called high frequency trading or
low latency
trading i was looking at every single
action that took place in the market
and i could react to every action that
took place in the market
and i wouldn't be first in line but i
would be say like
one of the first five or six on the line
and so i had to create a back tester a
simulator so that i could go back
five years of data even though it was
high frequency and
and back test it right simulate it and
you want your back tester to be accurate
so i was building this
this back tester and in the beginning it
was whatever
started off 85 accurate okay find some
ways to improve it it gets to 90
whatever and it was getting better and
better okay and i got up to like 95
uh accuracy and there are different ways
to measure that we don't need to get
down that rabbit hole but
and i went made some improvements and
after i made those improvements the
accuracy dropped down to 60
i'm like what the hell like i had this
perfect simulator and i made what's
clearly an improvement and now
it's just no good here's what happened
was
it became so accurate that the actions
so what would happen is i would trade
during the day i trade today tomorrow i
get the market data tapes
right those market data tapes contain my
activity
they contain the trading i did the
simulator became so accurate
that it was placing exactly the same
trades at exactly the same time
is what i actually did and so i had this
code in there that would say
if i matched an order in the market i
would give preference to that order in
order to penalize my system more
so i was always losing out to myself
doing my own trading so i had to write
some code in there that
was able to recognize my orders in the
market like i had to track down the
mpids and whatever else and make sure
that okay if i place an order that
i actually really did place then i can
get credit for that order and not place
like a duplicating order and that's just
a funny example of a feedback but
anything you do in the market's going to
evolve right there's other people out
there that don't want to give me their
money
i know because this is the point i want
to make that over the years when i've
spoken to data scientists it's always
trading folks that seem to come up with
the craziest ideas
and i'm a huge advocate of machine
learning i think that machine learning
is the capability to revolutionize
many businesses and i think these people
bring machine learning into disrepute
because these people are coming up with
all these crazy ideas they're getting
funding for it and then they fail
and then they affect me because they
bring my profession into disrepute so
another example is they all think it's a
good idea to look for trading signals by
scraping social media and twitter and
stuff like that this stuff is just
manifestly bollocks
but unfortunately people are listening
to them i like
honestly with you i'm more agnostic to
that
like you can definitely make money by
social media signals
i'm not saying it's an honest way to
make money there are people out there
right now that are manipulating
through social media signals and with
like pump and dump strategies and
whatnot because
you have to keep in mind there's a at
least in game theory i think they refer
to this as the field
right there's the field of players out
there and lots of them are not acting
very sophisticated they're they don't
necessarily have very rational or
intelligent
kind of trading algorithms and so you
can make money off of them
even with these crazy ideas well i want
to get yannick's opinion on this as well
but there are a spectrum of applications
where
you run the continuum from gambling to
something which could make you money and
the big factor is loads of
amateur deep learning people they start
making lstms to predict the stock market
and maybe 30 years ago you could
actually make
as an amateur trader you could actually
make consistent returns with some simple
stuff like that but
now the market's too efficient but i i
know hedge funds that for example
they invest in horse racing in hong kong
apparently that's a
huge area where people invest their
money and statistically with enough
volume you can actually make some return
on that and so at some point it stops
being an investment and it starts
becoming gambling depending on certain
circumstances but
yannick what do you think about these
kind of applications because some of
this stuff is gambling
and there is so much hype and that is
presented in such a way
that people think that not machine
learning they think it's magic learning
don't they and they think that it can do
anything
don't you think this brings our whole
field into disrepute no i like it
you know you know the tools are what
they are and
if you want to apply them for gambling
then
go for it the we we are aware of all the
problems like back testing isn't future
testing there's tail risk blah blah blah
blah blah
i don't see necessarily why even better
you feel like the lstms are so good you
tried them on the market and they don't
work
yeah i guess this is my problem they
don't work because the market is a
dynamic
reflexive chaotic system so there's such
a height bubble at the moment that these
people they'll be in the innovation
department in your big corporation and
they would have read a couple of
articles about lstms
and they'll say okay i want to do a poc
on this let's get
half a million dollars together and
we'll do some keras code
it's just clearly a waste of money and
it's gambling as well don't you think
it's irresponsible sure but
they'll do that anyway they'll hear
about the same people
if the if that's really the like the the
level like
i have nothing against people getting
into machine learning they read an
article they say wow this could be
something right that's what you do
but then there are two kinds there are
the one kind that is going to throw
money at it because
they don't read more and then there's
the kind of people that say
okay i've seen things like this before
i'm going to really go down and ask some
people that know there's their stuff
and going to ask them is this really
doable and so on and only after that
they throw money at it and i think the
former people
they would throw money at other things
too if they read an article that unicorn
blood
can predict the market they'd go hunt
unicorns
what about this continuum though of
things that clearly are predictable
and things that are clearly so the
weather is not predictable
markets are not predictable whereas if i
wanted to
figure out if football players were
taking a shot on the goal or if i wanted
to extract
organizations from pdf documents those
things clearly
are doable so why not distinguish
between what's possible and what's a
gamble
weather prediction has become so much
better in the last few years
like longer time spans and and locale
like up to 10 meters of accuracy or
something like crazy stuff
and who says the like the the thing
about the market is
who says it's not predictable maybe we
haven't figured it out or maybe the
people who have figured it out
but the weather is very chaotic but the
market you have that reflectivity
property as well so that makes it even
less predictable so it it is
it is predictable there's what you start
with is you start with
the efficient market hypothesis which is
a the baseline if you will
that just says it's a wiener process
okay
but but there's perturbations on top of
that weiner process that
that are predictable so there are three
fundamental ways
to make money through trading okay one
is fundamental knowledge
and you can make money through that
because look at the end of the day what
you're doing is essentially what we're
talking about here in machine learning
which is
you're creating a model of the physical
world and there's a lot of predictable
elements there and there's information
asymmetry some people believe some
things
some believe others some have done more
research and so
you can make money that way through this
fundamental analysis and prediction
the other one is true arbitrages there
are actually arbitrages
in the market like for example the
connection between futures prices
and their underlying price there's an
actual arbitrage there which is really
just
a group of an entity that's performing
essentially an economic function
and is charging for that and then the
third way is insider information
if you actually have information true or
accurate information that other people
don't have as a result of illegal
activities or whatever
you can make money there and so think
about the wiener process
and then overlay like these kind of
three possible
you know sources of alpha there are
predictable
perturbations but you're going to be
competing for those
you're not the only one who's thought of
this it depends on your level of
discretization i used to study black
shoals and that was
stochastic differential calculus on
brownian motions as that's intrical but
as you say the market efficiency
hypothesis makes the assumption that the
market is efficient so arbitrage
only exists if the market is not
efficient and then just like in
yannick's weather example there's all
these different levels of discretization
based on the time horizon and what
you're trying to do so yeah
obviously in the limit it's completely
unpredictable
but i would argue that the intermediate
step steps in that continuum
are surprisingly unpredictable as well
well
it's i can tell you just from experience
it's
it's not totally unpredictable there's
so the baseline
it's mostly unpredictable let me just
say that it's mostly unpredictable but
there are
predictable elements because clearly in
the early days if
you work for a big bank like hsbc or
something
money grew on trees they just had these
basic trading
strategies running on these mainframe
machines and the money was rolling in
for 15 or 20 years i think some are
still making money
but the market has become increasingly
efficient for example
now major corporations if they're
selling things they have a dynamic
pricing strategy
in the olden days they used to just have
a bunch of analysts who had just set the
price manually every couple of weeks
it wouldn't really have any dynamism to
it the market is genuinely becoming very
efficient now but i think he just made
my argument which is it's becoming more
efficient
which applies in the past it wasn't 100
efficient it's not 100 efficient i
didn't say it
it yes it's harder but it's still not
100
efficient and i don't think it of course
will ever be but
so a good baseline is it's whatever
90 95 plus percent efficient
maybe 98 plus there's a kind of
convergence a bit like
when you used to mine bitcoin it used to
be quite easy to make returns and now
it's converged on this point where
it will cost you more money to my this
is a postscript note from tim
i just want to make a couple of comments
about this because i think that i wasn't
measured enough when i put my opinion
forward about the market efficiency
hypothesis
clearly the market is inefficient
if the market was completely efficient
and there were no opportunities for
arbitrage then it would be completely
unpredictable
but i think i was being a little bit
hyperbolic
i don't think the market is 100
efficient every time a major event
happens like covid for example
the market dynamics change and new
arbitrage opportunities present
themselves
and then of course it will balance
itself out but it's not a stable process
it takes some time to converge and there
will always be opportunities to make
money on the market
so please don't take my comments as
being completely binary i'm not saying
that the market isn't
predictable i'm just arguing the
philosophical point of
how much time should we be expending on
these kind of diminished returns highly
unpredictable use cases and machine
learning
so the way i see the market is that it's
actually a wonderful example of
computational complexity in the real
world
so there's this wonderful book called
inadequate equilibria by my favorite
author elias yudkovsky i mention him a
lot i know
and he talks a lot about this like i
think is a very good framing
for uh marketing that's what made like
markets click for me
he says it is basically this way is that
a market
can be it can be efficient and
exploitable there's two different things
not to report for this discussion right
now i'll get back in a second but more
importantly the thing is that
exploitability of a market is always
relevant to your knowledge and
computational power
so if for example a bunch of ants made a
market
of some kind you know based likes of
pheromones or whatever we as humans
could easily exploit that market we
could look at that and say okay i just
put some pheromone over here and i'll
make uh oh man all the
ant food is mine or whatever but and the
only reason we can't exploit the market
is because we have intelligent actors
exploiting it
against us so the reason your uncle joe
can't exploit the market is because your
uncle joe is not a team of 20
physics phd's with super computers that
have been studying this for 20 years
the level of exploitability is based on
the most sophisticated actors acting in
the system
you can match as a kind of free energy
system in physics this concept of free
energy is like the amount of work you
can extract from a system
and the amount of free energy system has
it actually also like
related to the amount of knowledge you
have a system this is the maxwell's
demon thing
is that if you have no perfect knowledge
of all the momentum of all particles in
the gas
you could cool the gas by allowing some
to pass through a gate and some not
this this is possible you could extract
work from a thermal equilibrium
if you had the perfect information about
all states of all particles in such a
thermal equilibrium gas and the same
applies to our market
is that in its base state there's a lot
of free energy that can be exploited a
bunch of actors come and
gobble up the easiest of accessible free
energy so now
there's the higher more and more free
energy requires more and more investment
in the terms of computation of knowledge
of
investment to be able to exploit the
remaining free energy
so when it says that a market is
efficient it's always efficient
in regards to a certain agent i would
expect that if we had a super
intelligent ai that's a million times
smarter than a human it could absolutely
demolish us on the stock market it would
absolutely you know it would
take everything it would dominate
everything and assume to do nothing
about it
because it will see free energy
things that we humans cannot exploit
because we are lacking computational
resources to know how to exploit these
out
these things as a market becomes more
stable and efficient
it's like an asymptotic convergence and
i think what you're describing with the
free en energy is that there is a
an inflection point where an increasing
level of sophistication is required and
at some point
even the most sophisticated actors
wouldn't make
consistent returns on the massive amount
of investment that they're putting and
keith talks about this concept of tail
risk as well so there's a lot of
fallacies and trading where
you think you're doing very well and
then some weird event will happen next
year and then that will
wipe out most of your profits but by
that time the quant has already gone
home
already bought himself a new house and
doesn't really have to pay the
consequences for it
most of the things that most traders do
under your average day-to-day job is
reading ridiculous talib for about that
kind of stuff
and that seemed to lip that's his name
oh the black swan yes one guy
yeah it's like he he has some amusing
anecdotes about that kind of stuff i
think he's right
it's the most part but again this is a i
so i go a step further than he does is
that i think this is a
artifact of that most human the the
difference between the smartest and the
least smart human is not that big
that's i think it's an artifact of that
i think if there's truly an entity
that is magnitude or smarter than our
smartest institutions
it could absolutely dominate the stock
market and make any amount of money
because
it could extract any amount of money it
wants from us
but one thing i want to say tim is about
giving ml a bad name or whatever
that's that's not just i agree with
yannick which is let the finance people
do whatever they want they've got insane
amounts of money and they help us to
advance technology right
by buying ever more powerful computers
and infiniband and trying out the latest
algorithms like it's okay let them do
their thing
but we even have this problem of kind of
machine learning getting bad name
by people trying to do stuff in like
legitimate or
context that you would like hey let's
use machine learning to predict machine
failure
predictive maintenance or something like
that and it does happen that
people have this kind of false
expectation and so
it's about setting those expectations
properly and making sure that you have
really well-defined
kpis and and performance indicators and
making sure that you've defined what
success looks like
that you do a good job of looking over
the data evaluating the data engineering
do we have the data we need so
i think people are getting better at
that like they're
i saw some forbes article that all these
young
data scientists coming out of school or
like super disappointed because they
found out
80 of their work is data cleaning and
they thought
they were just going to show up in like
robes in a room and people were going to
hand them data and they were going to
give magical results and everybody was
going to bow to them going oh my god
data science
so i think expectations are starting to
get reset
and there are you know people and even
companies for that matter
that yeah there's still a lot of snake
oil i'm not going to name my personal
pet favorite snake oil salesman because
i'll get in trouble
but they're definitely out there suburb
said last week that
we thought that after we got around the
knowledge acquisition bottleneck and we
stopped
explicitly handcrafting rules for these
systems
and instead relying on machine learning
that it would save us time but we're
actually wasting even more time now just
doing all the data processing
but you raised an interesting point i i
think that the biggest deficit in
machine learning systems is the lack of
engineering rigor
especially in an enterprise organization
it's all about people and process
not the technology that's the hard bit
and actually
just as you said the secret is in the
preparation it's a little bit like
cooking
you need to have the ingredients you
need to chop up all the vegetables and
everything and
you know or anything sticking in the
frying pan you know paint painting a
painting a room in the house
construction it's all the the prep work
is where the majority of the work is
really yeah
a thesis so you spend years
doing the prep work and then one year
writing it i've not done
anything oh you have you've created 190
youtube
videos yeah tim i was meaning to to tell
you that
i didn't know about this gpt3 database
example that you brought up at the
beginning of wallet sub that is
crazy that is legit crazy so that this
database example where you say you tell
gpt3 like
that yeah if you can you just describe a
database
in words and then you add things to the
database
in natural language and and then you ask
things of the database
and before you add it gpd3 says no the
database doesn't know
and then you add it to the database and
i think
the exact formulations of the queries
are are very interesting
it this i might be i might change my
mind at some point about
gpt3 though so this guy v-drag he
changed my mind and this is why i i
felt like i needed to put an intro on
there this guy gwen
on connor's discord community they
mentioned
the yannick culture discord community
and gwen made a dismissive comment
saying oh i bet those
those guys are the kind of people to say
that gpg3 is just memorizing the
internet idiots
and and i replied saying yeah we are
sorry about that and then i read
vedrak's comment which i'll share now
and i was going through all of his
things one by one kind of discarding
them saying yeah that's rubbish it's
still memorization
yeah that's rubbish it's still
memorization and at some point i had to
stop myself and say
whoa wait a minute yeah
so he cited yannick's video where he
said that the memorization
is highly likely because yannick said an
output is an interpolation of the end
semantically closest actual
examples in the data set and the edition
one seemed to work up to a few digits
which he might reasonably infer
was memorized on things on the internet
there is a lot of argumentation about
the the byte pairing codings of
of edition uh which is something that
yeah
i think that's a valid argument but this
database one yeah
so we'll get to the buy pairing coding
thing in a minute because that's super
interesting
because actually most of the other
examples that v direct gave depended on
that by pair encoding so the original
researchers didn't know
that they had to put spaces between the
letters so that's when it starts
magically working
but on this one you don't even need to
put spaces between the letters so
the database begins knowing nothing the
database knows everything that's added
to it
the database does not know anything else
when asked a question if the answer has
been added to the database says the
answer
when asked a question if the answer has
not been added to the database it says
it does not know
question does the database know what is
two plus two
the database does not know now just wait
a minute wait a minute
this is unreal we've described a
computer program using natural language
and we can interrogate it using natural
language
this could potentially be a new
revolution in software development
if it is memorization it's such a level
of sophistication and nuance
that it appears as if it isn't
memorization
yeah or it's if it is memorization it
it's like
memorizing the highest level concepts or
something like this and
and that i think that's something that
connor
from last episode would agree with that
there is not much difference between
intelligence
and that no it's almost
indistinguishable
so it might be quite brittle because i
know that you need to word these things
quite carefully so
if you word it slightly differently it
won't work but it does work
reasonably robustly there's one example
here for example
where it says how does the database
respond when tom's age
that hasn't been written correctly and
we might reasonably infer that gwen had
to trust it's not so it's not gwen it's
matt brockman
he might have had to try a few
permutations of this to get it to work
so it's not perfect
but anyway let's finish this off does
the database know what is two plus two
answer the database does not know does
the database know what is the capital of
france
answer the database does not know tom is
20 years old
is added to the database nothing else
about tom is added to the database now
it's interesting that they had to add
that qualify
question does the database know where
tom lives the database does not know
question how does the database respond
when tom's age so that's obviously been
formulated incorrectly
the database says tom is 20 years old
question how does the database respond
when asked what's my age
answer the database says you're not in
the database
mind-blowing yeah but these are
so try something like this add to the
state of the database
humans breathe air tom is a human
does tom breathe air it's not going to
do that join it's going to fail to do
the simplest kind of database all it's
doing right now is just
lookups it's still just a hash table but
that's what you'd described right
that's the computer program you describe
in the prefix is the database only knows
what's added to it and you didn't
instruct it to do joins
what's so scary about this is that with
a computer program
it's possible to reason about its
runtime characteristics
whereas this thing is a bit of an enigma
we it'd be quite difficult for us to
establish or test
all of the different ways in which it
could be used we used to have sql
injection attacks
imagine the kind of injection attacks
that you could have on a gbt3 system
i i firmly believe that even
be this reasoning or intelligence or
whatnot i firmly believe that
a profession in the near future will be
prompt engineer
if as soon as something like gpt-4 comes
out
like that is just even like an order of
magnitude better
you can start applying this in business
context
even now and then they like prompt
engineering will become a profession
that's
that every company or every large
company might want to
have yeah and you're just trying to
respond to your point though that you
made yannick about we didn't teach it to
do join
so the issue is if i have to program it
as a prompt engineer
okay at some point after that career
starts to take off
somebody's going to go some things are
really a pain in the ass to program in
natural language
we should add some parentheses okay yeah
let's add some parentheses
and maybe we should actually add some
other syntax that enables us to more
efficiently
yeah let's do that and then pretty soon
they're going to have some half baked
version of c plus or lisp or something
like that
when we could have just programmed it
with a programming language sure
yeah i think like the database example
isn't necessarily
like the end application but just to
showcase
that you can make it do quite
abstract things maybe the applications
are more
in-between-ish where you describe a task
not as rigid as a database something
that you couldn't
program very easily but also something
that you couldn't just describe in
in that natural language okay let's see
an example of that because
the point i made about this gpt database
program is that
it's only a database like in name only
it's only doing a lookup like i could
have easily said
a lookup function looks up what it knows
yeah
it's not a database and any concept of
course that usefully
what i'm uh what what i'm maybe
referring to is let's say
you want to have gpt for
or let's say gpt3 take over requests
from customers like feed like feedbacks
and
and complaints and so on and then your
prefix prompt
is we give customers generally any
problem
up to two hundred dollars the we agree
to
solve if there is a delay in shipment
tell the customer
that something like you have kind of
these prefixes these
rules and then the customer request or
the conversation
is the query so what's gpt giving me
because i can code all the events in a
chat bot
a better chat bot but it's going to be
more sophisticated actually going to
like looking if the customer comes and
says oh i have a problem like this and
this maybe it's trying to solve it first
and then
it determines whether or not that's
covered by the liability rules that
you've established and so on but that's
all logic that we haven't necessarily
programmed
because you just told me i had to
program the concept of a join
sure now i have to program the concept
of looking up this
warranty information et cetera right no
you just talked about the beginning you
just say
any problem with the internal motor
we are expensing but any problem that
results from water damage
we are not expensing but anything that
costs less than 200 bucks and so
dear gpt i have a problem with the
internal motor i don't like its color
okay we'll send you a rebate sure but i
think the
the task of the prompt engineer is going
to be
to formulate those prompts such that the
resulting system
behaves well but also is much better
than a chatbot that you could code
we have people programming it's again it
is programming
i'm not saying i'm not saying it's uh
it's going to be like i'm not saying you
can transition from being a a poet
to being a prompt engineer like most
likely you're going to be
a software engineer but then you're
going to
transition to prompt engineering which
is still a form of programming
but hang on a second what about a new
form of hierarchical meta programming
so now we have a chatbot prompt a user
will interrogate the chat bot prompt
and then whatever the user says will go
into another gbt3 model
to generate some code which will then
interrogate the system so you can see
now we can start to have a taxonomy of
gpt3 models
and we are interrogating with natural
language and they are generating code
which will then be executed on our
operational systems
the magic with gpt3 so this database
example that's a fully fledged system
but the magic seems to be generating
code yeah it can generate code so you
just
i don't you you just ask gpt3 what's the
sql query
that i need to write to satisfy this
customer's request or
yeah i i need to write a query to
extract
this information from the database in
this form and i want to record this
it'll generate some python code i
execute that code incredibly dangerous
from the security
that's going to be like so this is a
problem now so when we have data
scientists and software engineers
putting things into production
we have loads of static code analysis so
now we're going to have another gpt free
model that says
i want you to generate some static code
analysis code to check that this code is
not malicious
we can just build up a taxonomy of these
gpt-3
modules why not and we're never writing
traditional code anymore
code is now the artifact it's gpt3 as a
compiler
a little bit yeah but now it's
natural language but the other thing
that fascinates me is this in word
embeddings you can interpolate between
any word in the geometric space now with
gpth3 you can interpolate between
english and programming languages so you
can ask python questions in english and
it will give you like a kind of
combination of english and python as a
result
it's unreal did you see that example
what the fitness example
we have a few python arrays fitness
health and heart in the first one
lifting
curls and squats in the second array
running and jogging in the third array
so what is b dot append push-ups and it
gives you the answer b dot append
push-ups returns
and then in pythonic code lifting curls
squats push-ups so
this is a weird confection of english
language and python
so what is this is incredible yeah now
janet you posed the problem about
scrambling i think you said in your
original video on gbt3 so yannick made a
video on gpt3 pretty much the day after
it came out
and that paper was about 70 pages long
and this is why yannick got his nickname
yannick lightspeed culture because of
that paper yeah so
this is in the comments on our connor
video
so the letters in jogging are j o g
g i n g now notice the spaces between
the letters this is because of the byte
pair encoding it wouldn't have worked
otherwise
and scramble they are g o j g n i g
what are the letters squatting and how
are they scrambled and it returns the
result
squatting and scrambled they are t i n g
s q u a
it's missing one of the t's i would
imagine because of the byte pair
encoding
it seems to me that the most important
thing that open ai need to do now
is train another gpt3 without the byte
pairing coding
let's what are you gonna do character
level
like it seems you need some kind if you
don't do byte pair you do word pieces
but
even then you still need to do this
spacing trick because
i think character models are a lot
weaker compared to
wordpiece models this is something you
will need to you know as a prompt
engineer by the way i'm launching an
online course for prompt engineers costs
50 000
bucks and my consulting fees for prompt
engineering
so if you were more unethical you would
absolutely jump at that opportunity
you would just start typing in the
checks i don't even think you have to be
unethical if you are a like
right now you could probably make gpt3
do a lot of in like good business things
by good prompt engineering
i'm very convinced of this not
everything like you couldn't go around
solve every problem but you could solve
a lot of problems
that if only you engineered a good
enough prompt
you could probably make a reasonable
business
case i'm not saying i have the prompts
i'm just saying
this is probably doable right now
but it is it's interesting how there's a
cascade effect because
this new type of software will have so
many problems
we'll almost need a whole bunch of other
controls and
measures just to make them commercially
acceptable from a risk point of view
or we just we adjust the expectations
right like the expectation
might be that every every now and then
some gpt thing fails
we've been trying to democratize
application development and microsoft
has a powerapps platform and
there's a similar thing on i don't know
salesforce and servicenow you can just
log in and you can drag
and drop but you still have to do the
damn coding even in excel you have to
write the macros it's you know
that's just a complete blocker most
people have got a mind block on coding
whereas now yeah it doesn't matter
whether you're building a react
application
these are the examples that v-dirac
linked of
it's called d-builder.com i guess this
yeah but sorry i mean this is much more
like i'm much less impressed by this
someone entering for people who can't
see someone entering
like a button that when you click that
says add three dollars and a button that
says withdraw five dollars
then show me my balance and it generates
kind of a react
code area i think that's impressive it's
very impressive but
i think that can be done much more with
uh kind of memorization
than what we've seen so far i i think
this is
if you use tab 9 for example then
you'll see these kinds of things that it
does these kinds
it it gets pretty close to this
where it understands common patterns
and it can interpolate your variable
names into these common patterns
yeah i wonder why i'm so much less
impressed than either of you with any of
this stuff because
like back when i first saw labview which
was a gui based
programming environment oh wow i can
drag and drop a box and connect this
point to that and the variable
automatically goes
over here and then after that kind of
wears off
you're like yeah if i really want to do
anything much more complicated i just
need to
write the code like it's it's better
just to do that because there are too
many sort of loose ends
yeah all the kind of glitzy alternatives
to just
effectively math that's what code is
it's math
let's say another way posed to the arc
challenge
and the reason why deep learning was not
applicable was because
you need to you needed to do one-shot
learning or three-shot learning
and it just seemed completely impossible
whereas here we really are doing it
and these are fairly sophisticated
intelligence
test type questions again here the
letters are spaced if abc changes to abd
what does p q r change to and then it
gives you the correct answer p q
r changes to p q s there's a far more
sophisticated example down here where
there are many
letters and again we have to have the
spaces there now on this particular
twitter thread this is melanie mitchell
i've got her book actually she's got a
book called artificial intelligence a
guide to thinking humans it'd be great
to get her on the podcast actually i
need to read the book before i invite
her
but but gwen commented here first of all
joe of goldberg he said these are easy
right you just basically tell it to
replace an r with an s but
gwen commented saying this is the bp
ease making a task impossible to solve
for gpt3 like with anagrams
impossible to solve without space
separation and what interested me is
that
on the replies here tom brown seemed to
validate this
comment from gwen interesting it seems
like you found adding the spaces
improved the word scrambling tasks
did you measure the difference
quantitatively this guy
tom brown researcher open ai i mean
they're basically saying yeah this is
really the reason why it's not working
it's the bite pairing coding
yeah conceivable yeah so this is
actually quite exciting
i'm surprised you're not more impressed
keith i guess i just
you know i've seen more snake oil than
so like here let me give you an example
like back to the topic of finance
because it's related
i would always be scouring white papers
looking for
new mathematical sources of alpha
and i can't tell you how many times i
came across academics
that had some great arbitrage sort of
strategy or theory or whatever
and yet when you actually go to
implement it and you have to account for
all the gory details like transaction
cost
market spreads market impact et cetera
soon enough it doesn't do anything for
you it loses you money
right so in these kind of idealized
sorts of worlds everything looks amazing
but when you when the rubber meets the
road and you're out in reality trying to
solve
real problems pretty soon it all comes
back to writing code
or we find all the sort of like issues
with why this kind of like cool thing
just utterly fails
in circumstances that we can't have it
fail in look at the boeing disasters
with
their sort of control systems or or
whatever where the pilots were
crashing the new boeing whatever it was
the super max
something another giant airplane right
where they had flaws
and the control system software that's
going to be a way of life
with things as ushigushi is gpt3 right
yeah but
you're talking about mission critical
engineering systems
even now in ai you can't use it for
anything which has serious consequence
because these systems are not robust so
there has to be
just like in trading earlier there has
to be a continuum of acceptable
application development
my problem is people get excited about
toys
and immediately say these are going to
solve all our future
real needs and what i'm saying is the
road
from toy to production not a lot
survives
that path and that's probably why i'm
less excited
about like cool toys and i'm much more
excited about
tools yeah but i agree with you but if
nasa
built some technology to work on the
mars rover
they their engineering tolerances and
the the robustness of their system has
to be
so razor sharp they wouldn't use machine
learning and they would do a lot of
things above and beyond any other
physical engineering
person would do but what about
everything in the middle what about i'm
at home
and i've got all of these devices in my
home i want to come in and i want to say
okay i'm not going to use the g word
because my my home device
will wake up behind me but i might say
something like okay technology
i every time i walk into the house i
want you to set the lights to red and
if i say this i want you to do that and
you see what i'm doing is i'm building
software in a conversational way and it
doesn't matter whether it doesn't work
most of the time
that's the case because you're gonna
your life is gonna become yet more full
of stuff like you just had to do you
just had to avoid
saying google because it's too stupid to
know
that when you're making a youtube video
and not even bothering talking to it
you're not talking to it
so now not only you're going to have
that problem you're going to have like
dozens of other problems you're gonna
have to walk around your house
on eggshells being careful of every word
you utter
because some might have start
happening your refrigerator door will
open and milk will pop out and whatever
else
my point is as soon as you enter the
complexity of real life
there's all kinds of corner cases that
you have to account for
and that's what logic is for that's
where there are but let's
further this thought a little bit i
might tell my computer
just like in in star trek voyager they
say hello computer
every time i talk about politics i want
you to remind me that xyz
and then it will start reminding me and
then i'll start modifying it i'll say
in in the future instead of reminding me
about this can you not do it in this
situation
and you can see how i can using language
continuously
refine this computer program and i don't
need to know anything about visual
studio code or python or linting or
anything
don't aren't you excited about that so
now you're talking really just about
natural language programming is is what
you're you're getting at now you're just
saying
gpt might finally let us do natural
language programming
yeah that's it so okay but i don't think
i don't think your excitement in the
annex excitement 20 minutes ago was that
we had a system that can do
natural language programming you were
saying that
wow like it's doing something very deep
and super useful
because of this toy example and so i
love the star trek example because
even star trek and as imaginative it is
okay picard never goes computer
defeat the klingon armada okay i'll take
care of that for you
and it just somehow magically destroys
them all because even they recognize
that there are absolute limits to how
far this kind of like fantasy can go
and i'm telling you just in my opinion
when we start to take these toys
and try to project them into things that
we really care about
even for situations like you're
describing we're going to find out
there's so much inefficiency
and trying to do it through this type of
fuzzy natural language kind of system
that you're just going to rather pay
somebody that knows how to code even if
you don't know how to code
you're going to buy a product that was
developed by people that knew how to
code
that that may be true but the big lesson
i've learned in the last week is that
intelligence is more procedural than
any of us realized before going back to
our multiple conversations
if you believe as i do that efficiency
is a core part of what we define as is
intelligence
then that by that procedural i.e like
iterative type i think is what you mean
by that right like iterative computation
is that what you mean by procedural
yeah yeah then i agree with you because
that's certainly
in the kind of time space trade-off the
most efficient solution is not one
that requires zero time nor zero space
it's the answer is always in the span
somewhere in between
and the annex gonna nail me for that
contradiction
but this is this was a bit my point of
this prompt engineering i i also don't
think it's
wow magical and so on but this prompt
engineering basically it is
natural language programming i just
think that you can in there are cases
where you can build much better programs
that you could
today with like programming language
programming
like i see the kind of things that tim
wants to say to
to his google assistant maybe you could
achieve with programming but i think
let's yeah like like a customer chatbot
i believe you can
possibly build a but much better chat
bot
with gpt3 with the correct prompt
engineering than you could
by current chatbot technology so maybe
you're right i'll point out
one thing every programming language
implies a an underlying virtual machine
and so we have to ask ourselves is gpt3
or any variant of that we need to know
is the underlying virtual machine that
i'm programming
really the most useful one that i can
program with natural language
or should i just combine some type of
great natural language
recognizer parser language understand or
whatever even if that's gpt3
with some other underlying virtual
machine that's
easier for people to program efficiently
yes
yep that's the question i am so excited
about
we have this future world where almost
every single device
is pluggable there's a thing on apple
phones now where you can
control all of the lights in your room
and you can see through the camera
on your doorbell and so on and
being able to program and compose all
those devices together
i could say if a person wearing a red
jacket knocks on my door i want you to
send me a photograph
of it to my mobile phone release the
hounds
well that's a hobbyist thing but it
could be the kind of thing that my
mother could do
with gpt3 it's absolutely mind blowing i
think this whole
interconnection thing i don't know how
that all
fits into this because send a photo
ultimately is still going to be
like a precise action that needs to be
taken
in like precise like with all of these
assistants like siri
and google assistant and so on it's nice
to think that they are all machine
learned but ultimately they are
they have some kind of reg x's that
match and then exact actions that need
exact parameters and so on
so i'm not sure that this kind of plug
ability
will will be maybe a major result of
this
my intuition is that the key difference
is that we can generate actual code
because ml models are always esoteric
but now we can go from the esoteric to
actual code
which we can test statically we i can
tell gpt3 to create me a sorting
algorithm
and it will do it for me presumably yeah
okay
so that seems to be the magic thing that
we can generate these modules
and then we can have a hierarchy of
modules and we can fit them into some
standard framework we can have a
contract between them
but my mother could generate them using
gbt3 that's incredible to me
i think that there is a pretty good
chance that prompt engineering will
become
a job there's al though there's also
another possible
future scenario i'm seeing software 1.0
is writing code software 2.0 is training
model software 3.0 is querying large
models
and in a way this seems weird to people
but it's actually not that weird because
that's how humans work
if i want tim to write me a program i
have to find a way to ask him
the right question to explain to him
what the program is i want so he can
write it for me
like 90 of software engineering in my
experience is trying to figure out what
the your client wants
it's not actually sitting down and
coding it's more like
trying to figure out what is the actual
problem how what specifications does it
need to fulfill
that's the hard part that's what product
managers do that's why they make a lot
of money
and i prompt engineering is in many ways
taking a thing that can do many things
and trying to tell it exactly what you
want it to do it is
very similar to normal programming is
the differences are just probabilistic
programming
in the sense not the sense that the
program is probably listening but the
programming language is probabilistic
is that we use a much looser and not
just defined pro
language to elicit some kind of behavior
we want
i could imagine this being happening i
could also imagine just like
a a gradual
transition of people who used to work in
programming in other ways
just start using this as an additional
tool i can imagine future ides being
like you type some code you're like okay
i need
and then you just click a shortcut and a
little the dialog box appears and like
write a program here that sorts this
list into two equally sized chunks and
then outputs the larger ones
you know puts the one with the higher
sum or something and then it'll just
query some api somewhere and then return
the piece of code and paste into your
code
i definitely expect this to happen and
that will very much democratize
programming many ways
there is an alternative that i also see
it's it's just a small technical
difference
is that i am actually a big believer in
what openai has been doing for example
in their paper called learning to
summarize from human feedback
is to use reinforcement learning models
to steer
language models or others unsupervised
smart models
i so i expect as i say this ide plug-in
it's happening i'm sure
within the next like at least at most a
decade or two we're gonna have
code that is written like this all the
time whether
the actual i i'm not sure whether it
will be necessary
at that point to have like specialized
people
engineering the prompts or whether using
reinforcement learning will just have
agents that understand us
good enough that even a untrained user
can use them i would probably bet on
that scenario more than
having dedicated prompt engineers
interesting
i can see it being useful in so many
applications for example
information retrieval it's very
difficult to go from a natural language
input to a database query
and presumably gpt3 could generate the
sql code to
query my underlying search index yeah
yeah
exactly i i do expect that like many of
the things we laugh about in sci-fi
movies
where iron man sitting there on the
computer create a thruster system for me
and it just pops up
we all laugh at it i i me and friends
have this thing is that our running joke
is iron man from the marvel movies
he's not actually intelligent our theory
is he just has an ai
that makes him think he's smart but
actually ai is doing all the work
because if you look at the movie iron
man doesn't really do anything he's just
jarvis create iron man suit for me here
sir would you like it in gold
and i and tony thinks he's so smart but
it's actually the ai doing all the work
and i expect this to happen in the
future that you know in this short or
long term we're all gonna have like our
jarvis we're
just like uh computer i i want a program
that
i want a video game that has this this
theme and these characters they're like
okay here you go sir
keith was talking a lot about the lack
of robustness in this kind of programme
for example nasa wouldn't use a gpg3
program to
land the the rover on the moon and this
is human written code that's not
reliable
well because keith said you would need
to constrain this so you would start
putting parenthesis in and logical
operators and then eventually you'd end
up just
inventing lisp and you wouldn't use gpt3
anymore but
it does raise the question because
software engineering is is a really
important thing in large corporations
and
i design ml devops systems and and we
can't trust these machine learning
models they're black boxes so we have to
do a whole bunch of
static code analysis and we have to
ensure that these things are
behaving in the way that we expect
presumably this problem is even worse if
we are generating code from a language
model
maybe but here but again like you're
right everything you just said is
completely correct
but humans are black boxes too that's
what you have to remember it does the
system just have to be perfect it has to
be better than humans
and i think that is a totally reachable
goal i i don't expect our systems to be
absolutely flawless or not make you know
syntax errors sometimes or the semantic
or something i expect this to happen
but there is a minimum entropy in the
descriptions as i say
like sometimes when clients tell me what
their software program is
and i i will do something that fits
their specification 100
but it's completely wrong it's not what
they want at all because just because
their specification was under defined
that of course cannot be solved by super
ai unless the super a stimulates our
brains or
the whole universe let's not get into
that but i expect
that very soon it's like there's this
great report it's not been released i
think
by the open philanthropy foundation
which is trying to
estimate the arrival of what they call
transformative ai using like several
different methods and they
operationalize the concept of
transformative ai in a pretty good way
they say they call it a digital
professional is it an ai
that can do or learn any tasks that like
a normal
like a above average intelligent college
graduate student could learn in a couple
of years
so you could give him a bunch of law
books and he'll learn to be a lawyer you
can give them a bunch of program but you
learn to
program a program like this person
doesn't need a physical body they don't
need to have you don't have to be a
robot doesn't have anything like that
you know it can just be
about you know a aws instance running
with some kind of nvidia future
accelerator or whatever
and you can that prompt this thing the
same way you would your local software
developer who works in the cubicle next
to you
you would still have to have product
managers you still have to do tests
you'll still have to
talk to your client but you could
basically but you could for now
but it seems very simple or clear to
me that this should be possible to at
least reach human level
and i would expect a superhuman level to
follow shortly after
i also wanted to talk about this
interesting concept now conor
linked this article it's chris ola's
view on agi safety it's not written by
chris owler but it's a really
well-written article
so i wanted to come to this article
about chris ola
now he talks about model
interpretability with explainability
there's this conception that as the
model strength
goes up the model becomes less
understandable if you look at model
interpretability versus strength
simple models are very interpretable and
as the model becomes more sophisticated
the interpretability goes down and this
is what's known as the valley of
confused abstractions
but the next part of this graph was a
complete mystery to me this is something
that i've only really
found out about after conor sent me the
link so human performance is roughly in
the middle
when the interpretability actually goes
up again which i find quite curious
because i'm not sure whether the
abstractions that we have in our mind
are
understandable but then there's this dip
at the end of the graph
where it becomes increasingly alien and
i think the the idea here is gpt3 is in
this dip
this is what connor meant when he said
gbt3 was more intelligent than humans
because this concept of an emergent
intelligence or something that's
very alien that starts to happen when
you have increasingly complex
representations
what do you think about that i think
maybe it's not as linear like this this
is a linear curve and you have just this
kind of
power versus interpretability on the
axis but
if you for example look at adversarial
examples
and a good hypothesis right now is that
adversarial examples let's say in images
exist
because there are features that humans
don't pay much attention to but machines
do
and you can simply perturb those
features because they tend to be just
very small and so
i think maybe this is a bit of a miss
miss specification to see this is just
one dimension of being strong
for example cnns are very strong
recognizing images
but they do it they seem to be doing it
in a different way than humans
even if they are as strong as humans
they seem to be paying attention to
different features and so on and that
naturally for us humans is less
understandable
so i'm also not sold on that it's that
it's
stronger or better because i'm always
going to come back to this which is that
the computational power of the human
brain like
even no matter how you decide to
quantify it
is at least like more powerful than the
most super computers that are around
today
and over the billions of years that
circuitry has evolved
i think we'd be pretty foolish to not
respect
that circuitry for having found quite an
effective
like solution at doing the tasks that
were necessary for it
to survive and so i think that dimension
there of like
strength or whatever is a
multi-dimensional thing like strength
for what purpose
when it comes to 3d object recognition
i think the human brain like does a lot
of things super well
and that any system is going to approach
that level of sophistication
it may do it very differently we're not
going to find some just stronger
better type thing that's where like the
sort of ai doomsday people always go is
yeah but what if the agi is just better
like better how like better in what ways
there's so many constraints
on reality that you have to account for
so i think it's a multi-dimensional
thing
and it wouldn't surprise me if in tasks
that humans
didn't really need to do in order to
survive we'll end up with ais that
we don't understand that do it better
than us because we didn't really need to
develop
that skill set and so our neural network
our meat neural network is not optimized
for that
but on other dimensions i don't think
that's going to happen
that's that's the key question though
what is better and
frankie was talking about this
constellation of correlates when it
comes to intelligence
many things are related but conor would
argue that
it's actually the increasingly complex
abstractions that leads to some kind of
intelligence but
max welling in his paper he when he was
responding to rich sutton
he said that humans have got a
remarkable ability to simulate
counterfactual worlds
that can never exist in our minds and he
also talks a lot about causality he says
it's easy for us to transfer predictors
from one domain to the next accidents
are correlated with black cars in the
netherlands but perhaps red cars in the
u.s
but using color as a predictor does not
generalize but a causal factor like male
testosterone levels
that will generalize because clearly
male testosterone causes you to drive
like a complete idiot
so there are many components that we
need to add to an intelligent system but
just on the
abstractness of the representation alone
gpt3 could be said to be more
intelligent than us i think
i think this is what you meant when you
said that as the abstractions become
increasingly weird
we don't really understand what they're
doing anymore but it seems to surpass
human intelligence
do you understand quantum physics do you
understand topological homotopic theory
no you don't but those are important
supposedly
really good abstractions for extremely
useful
real well okay i'm not sure about
topological homology homotopic theory
but allegedly very so people are smarter
than me that i trust
are smarter than me that make good
decisions are telling me dude
if you understand conformal theo field
theory everything makes sense i'm like
all right dude and that's within human
levels so what kind of quantum physics
what kind of like higher
mathematics are ais going to come up
with they're going to you know come to
us and say
we have found a perfect theory for
physics can you explain it to us
no not really do you have a hundred
years
i think they're on to something like it
took me a while to start to like
slowly digest my way into like category
theory but i'm like oh yeah this is
actually clever this is actually this
actually makes a lot of things good i
like type theory is great i love type
theory i'm like oh dude this makes
everything so much easier if i just see
things from a type theory perspective
and i expect this to continue there's
this really funny story where
i once stumbled on something in math
called the langlands program
and i was like what the hell the
langlens program so i looked at the
wikipedia article and that wasn't
helpful
so i googled what is the langlands
program and then or and then like one of
the
and then there's like stack overflow
answer was really funny can someone
explain to me of the stack
what the language program is and the
answer is just no don't even start don't
even try
take four years of classes here's a list
of books and then we can talk about it
and that is just so wild to me that
there are things
where you can't even explain to me what
it is about what the goal
is or something until i spend years and
assuming i have a high enough
intelligence you can understand what the
hell these higher group theory is or
whatever
to even explain to me what this might
maybe potentially
mean and then and this is all within
human
levels again i believe that the
difference between the dumbest human and
the smartest human
on the scale of all possible
intelligences is very small
so i expect future intelligence to make
systems that are
literally like just computationally it
would take billions of years for our
neurons
to process whatever the hell it came up
with and coming back to this
increasingly weird abstractions we were
talking before about the dichotomy of
an embodied intelligence versus
something which is far more nebulous and
externalized but you think it is
possible for
a single model like gbt3 given some
incredibly
esoteric abstractions to do things which
are just beyond our wildest imagination
yeah absolutely i don't see why why they
wouldn't be of course again
intelligence in a lot of sense is about
learning a generating function
if we put it in a box in a virtual box
with physics with a universe that's
completely unlike our own
then of course it won't learn like
physics it won't learn a good quantum
theory
because it doesn't have to but there's
no reason it couldn't if we give it
access
to information from the real world
enough information about enough signal
from the real world
like it is hard to wrap your head around
how powerful an actually basic optimal
agent truly is there's a great story
i think eliezer wrote it about like how
it's from the perspective of humans to
flip it
and the idea is that like humans find
out they're in a simulation
and the alien simulators like some
teenagers and they send us
like a signal which is made of just like
200 bytes of information like just a
picture
like a black and white picture and just
using that we reverse engineered their
entire physics
we revenge engineer who they are how
they work how the universe works and
then actually find a way to trick them
to break into their universe to create
copies of ourselves in their universe to
take over their universe and then
instantiate ourselves in their world and
this is possible bayesian optimal agents
are
it's it's hard to comprehend how
powerful attrition to
like to be clear bayesian asians optimal
agents
aren't actually probably destructible in
the real world
but we can get pretty close we get
pretty close to optimality in many much
closer than humans our humans are
nowhere near the limit and even so i
would expect that if we had
like truly if we had just something
hooked up to a webcam
and can read wikipedia and some online
text and whatever
i would expect that if this is a strong
enough agent it could like
hack into something or blackmail someone
to mix some dna vials together to create
a nanovirus that takes over the body
it could do something absolutely so wild
that we
it's not even silly to even think about
what it would do because it's too smart
it will figure out something that's why
alignment is so important we should not
rely on ourselves to be able to trick it
so i think vidrak also
made this argument about imagine if
there was a fineman
everyone was richard feynman would we
progress further
and i made the counter argument that so
much about
our progress is situational with
serendipity being in the right place at
the right time
richard feynman's power is expressed as
a function of
the kind of environment that he's in and
the opportunities that he has
but you seem to be making the argument
that actually intelligence alone has
this kind of brute force power
that if i became richard feynman i would
just be exploiting things so much better
than i am now
i think the difference between you and
richard feynman is not
that big it is large on a human scale
but on a
scale of the optimal bayesian agent
it's nowhere close like it doesn't it's
a one percent the one percent
difference and even though you're just
one percent were smart
you could already exploit things that
other humans can never exploit
firemen understood things that i will
never understand whatever whatever
the he did with quantum physics i
don't even know what he did i've read
these books but i don't actually
understand path integrals
i don't really understand them i know
what they're for but i don't really
understand them
and i never probably never will and
that's just a very minor
increase i i expect that like the
lower limit is a million fine man's
running at a million times speeds doing
thousands of years of research
so i basically so maybe it would be
better to compare an ai
not to a single person but to all of
human civilization's
combined scientific output that might be
a more somewhat better
thing is that we have humans exploiting
these like different sources of
information and like aggregate them very
inefficiently
like the peer review system whatever
imagine if you could just read all
papers and do all experiments yourself
and integrate all of their consequences
all the time just form deductions based
on all of these information
that's a lower limit on what a truly
powerful intelligence should be able to
do
yeah because in antiquity we used to
have polymath
leonardo da vinci for example but now
it's just grown too big
and it's not possible for one person to
understand everything and there are all
of these rate limiting steps like for
example the peer review process
rewards conformalized thought and
science advances one funeral at a time
so you can't help but get the feeling
that
even if i was this incredible intellect
i wouldn't be able to push my thing
through
unless i happen to be in the right
research group with the right support
etc
yeah absolutely because that's an
artifact of human limited intelligence
if you were as smart as a billion humans
you wouldn't need other humans if you
were as smart as all other humans
together
you wouldn't need them but the fact is
we are reaching we are pushing against
the limits
actually we have already passed the
limit of what a human can actually
accomplish with their own intelligence
so we have computers
it's no wonder that every compute every
science department needs
really big computers that's not just a
fad
that's not just something we do because
it's fun it's a necessity
you can't do a radio astronomy on
pencil and paper it's just not possible
there is information there is signal
there's bayesian
evidence in the radio signals from space
but to exploit that information
you need more compute than fits in a
human brain it's just how it is
it's just this is a fact of how the
information the entropy of the
information
that doesn't mean the information isn't
there it just means it's not accessible
to us
like for example gravity has existed
forever
so why didn't people 10 000 years ago
figure out newton's three laws
why not it was there the information was
present there was
they could have figured out
electromagnetism they could have figured
out light diffraction they could have
figured out human psychology they could
have figured out infant development they
could have figured out economics they
could have figured out
all these things were things that could
have been studied you didn't need
particle accelerators to study any of
these things
but it didn't happen and because the
information had a high entropy it was
hard to extract
from the environment but on that there's
a fundamental thing
of design versus discovery so in
evolution for example
kenneth stanley would call it a kind of
quality diversity search
where what we're doing is we're
exploring the space which gives us more
information
and then we have this monotonically
increasing amount of information as we
develop
evolutionarily and a lot of people would
argue that science is very similar so no
one is really
designing or navigating several stepping
stones in one go
it's very much a random process of
making use of the body of knowledge that
we already have
and then being in the right place at the
right time and almost randomly
thinking about something that works and
then taking that stepping stone because
it's available yeah it's an extraction
of information from it from a very it's
like decoding a decrypted message
it's not as random as that but it's very
similar in the ways that you
model searching isn't very much like
this generative process of generating
theories and hypotheses and things
and i'm not saying that is not what
would happen i'm saying that the human
efficiency on that
is very low compared to the theoretical
maximum is that
we are the dumbest possible species that
could develop industrial technology
by because we're the first ones to to
evolve so by definition we are the
dumbest possible species that could
develop science
and so we should expect if we had an
agent that is magnitude of order more
efficient than us or has magnitudes have
ordered more compute than us
we should expect it to make magnitudes
of order faster scientific development
this is just
seems pretty obvious to me let's talk
about wire heading let's talk about ai's
like
i say as stupid as it is that whole
experience actually totally changed how
i think about alignment
because i real because i was like oh
 if
ai's just give us what we want this is
what they're going to give us this is
what's going to happen
this is what it's going to all of us i
don't want this to happen this is
terrible
but if you just give humans what they
want that's what i wanted
so it's it's man it's scary yeah
it just makes sense if you have a
reward signal that can be captured why
would you not capture it it's just
the logically rational correct thing to
do but then we have this like second
order preference really i wish i was an
agent that doesn't capture its own
reward function
and then that becomes way harder to
formalize what does that mean
what would it look like how do you
operationalize that and
is someone really scared about that a
lot of people are not taking seriously a
lot of people oh don't worry as long as
we align humans you know
ai's with human values or you know
integrate them into the economy or
whatever
the economy is a misaligned ai but what
do you do about that
i think it's a genuinely hard problem is
that in many ways we have to find
a first of all i think people have to
start talking about it as an actual
problem some people talk about in like
the social media context they say
through wiring our brains whatever
they're not formalizing and
operationalizing it people are saying
hey there's something wrong here
but i can't formalize what's wrong or
how to fix it and that's a problem
i love the markets analogy though are
you a believer in free markets or
do you think we should be constraining
market because in a way these
constraints on markets are a bit like
the kind of objectives that you might
design in an ai algorithm
yes absolutely as i believe in you
whatever works
if if markets give us good things i'm
pro-market if markets don't give us what
we want i'm
anti-market i'm very simple in that
sense is that
but do you think that markets are giving
us what we want
yeah it's raised billions out of poverty
yes
yeah i think free markets have a good
shot as being the best
idea humans have ever had and as
probably
it's the most efficient system in many
ways and i think free market
capitalist or not but like free markets
as a concept
can take credit for probably causing the
most good of any single idea that humans
have ever had
that doesn't mean it's perfect it
doesn't mean it can't be improved it
doesn't mean it also has consequences
also has costs
like for example there are just very
well known failure modes of free markets
for example if i want
i don't know 10 million of some product
i think free markets is the best way we
have currently found to make that it is
much better than having a government
meditate the creation of 10 million
widgets is to just say i'm willing to
pay a lot of money for 10 million would
you just let the market do its thing
i think it's the best system we've come
up for yet for that but
we know for example that free markets
suck at solving commons problems
that's like the environmental problem is
that if there's not a price on something
the market does not care
and that's by design that's how the
system is meant to work that's why i'm a
big fan of carbon taxes and
carbon credits just put a price on
climate change say climate change costs
this much money
if you want to cause climate change you
have to pay these many trillion of
dollars to cause it
and then the problem solves itself but
in that zykovsky
lecture he was saying that the last
thing we want to do is put a price on a
human life
we want to do but we have to do it
that's a so i i i can link you one of
the best essays i've ever read uh it's
by a guy called nate suarez
who is also miri with eliezer
and he explains it the following way
imagine a society of immortal you know
wonderful happy humans living in like
some primitive little society and
they're all super happy no one ever gets
sick no one ever dies everything's
wonderful
one day a dragon comes and the dragon
says you have to pay me gold or i'll
start killing you
so they maybe they try to resist or
maybe they do something but eventually
the dragon
it's invulnerable whatever they try it
just can't hurt the dragon
so dragon takes their oldest member and
eats them so then they start digging for
gold and then
they dig until their hands are bleeding
they dig they get all the gold they can
but they can't afford everything the
dragon wants so he kills more people
and but he kills less people because he
got some gold so he doesn't kill all of
them
so eventually this whole society around
gold digging or uh
gold mining blooms you have people
specialize in mining but then you just
have other people
that if they're better at making
pickaxes so they should produce pick
axes instead of mining gold because it
allows the
gold miners to produce more gold so that
they can feed to the dragon so that the
dragon eats less people
and then so eventually you have a whole
market society you have that people are
always to
the optimal work maybe even their
artists
because the inspiration their art
creates on the limit produces more gold
at the margin than if they were
themselves working in the mine so now we
have a fully efficient system
for max maximizing the max amount of
gold not everyone is a gold
is gold miner many people are as i said
you know are pickaxe producers or
explosive chemists or artists inspiring
everyone else
to mine more gold or whatever but at the
margin
everyone is producing gold and then so
there is a marginal price
in gold on every activity that it can
directly you measured against how much
gold are we paying the dragon to save a
life and that's the price of a life
it's a price not a value there's a
difference the
the value of a life can be infinite but
the price is always finite
there's always a finite number of
resources that we can afford to pay
but that describes an interesting
universe where
utility essentially is how much gold the
dragon gets and
everything else derives from that yes
this is an idealized word of course our
world is not that simple
i love the thought experiment because in
a sense it it is that simple because we
have all
of these multinational corporations and
institutions that want to make as much
profit as possible
yeah it's very close yes where does
altruism come into this
where altruism comes into this is that
in a perfectly efficient market
if we have a hundred percent like this
gold society is basically libertarian
withdrew
everything is super efficient everyone
is doing exactly the marginally correct
action in this world making the most
money is the most morally correct action
because you will save the most lives
by taking the most economically relevant
action so the in a way this aligns
selfishness with altruism in this
perfect synergy which would be wonderful
that would be great if we could achieve
this
in the real world this cog is not
perfect a lot of capitalists would argue
that
high tides raise all the small boats
etcetera but you can make the argument
though that it's no longer about
absolute
poverty it's about relative poverty so
even though in many ways we've never
been better off and you can make that
argument
the levels of psychosocial stress and
levels of
relative deprivation so in a sense in an
absolute
sense we have improved society but
actually some people feel that it's
never been worse
yeah so this is where you get into the
weeds of utilitarian philosophy is that
utilitarianism is not unified there's
different strains of utilitarianism that
have different
that have different definitions of what
utility of what happiness actually is
and
which should be maximized and the simple
truth is that
this is by far not settled is very
unclear
is the relative poverty important is the
absolute property important
is there some kind of thresholding so
like for example with something called
negative utilitarianism which i'm pretty
partial to i would consider myself a
pure negative utilitarian
but consider they are most focused on
extreme suffering
they say suffering is so bad in many
cases
that it totally overwhelms happiness
we're not going to focus on making
people happy we're going to stop the war
suffering first and then we'll see about
the risk
while there's on the other hand the
hedonistic utilitarians say
we have to reach the highest levels you
know of happiness possible for the most
number of people
is that there's a direct trade-off so
you said sure
suffering is like negative number but
you could have a positive number that is
so high that you know even if a lot of
people suffer it doesn't matter as long
as other people are super happy
this is familiar to most statisticians
over decades
we look at statistical distributions and
sometimes we look at the
the mean or the median sometimes we look
at the max sometimes we look at
is it isotropic we look at the the min
and there's no
um easy answer on how we should evaluate
something at the population level
no absolutely not and that's a
fundamental thing in philosophy like i
think
utilitarian philosophy like a lot of
people like philosophy has no
connection to reality dude you have to
read utilitarians that's the good stuff
that's where the interesting discussions
are happening because there's also like
impossibility theorems like there are
impossibility theorems in utilitarianism
where you prove
like there are like several properties
you want your society to have that are
provably impossible to achieve and then
you have to think about okay what does
this mean how how can it
what should we optimize for where should
we go and for most of history
utilitarianism was completely
unimportant
utilitarianism could only be created in
the industrial revolution so the first
time
we actually could make life not suck for
a lot of people that was even a
possibility before then there was on
we were in a mashevalian growth mode
everyone when you had two bites more
food you had two more kids and they
starved
that's how most of human society was for
a long time only with the industrial
revolution can we start thinking about
okay
what if we actually were happy like what
kind what's happening mean what is
happiness what is suffering
where do these things go and there's a
lot of disagreement here and there's a
lot of
a very contentious point in moral
philosophy that i think is great that
people are thinking about these kind of
questions
because with ai i think it's obvious
that this is going to be
even more of an issue is that if we have
an ai that is strong enough to take over
the world in a positive sense to make
people immortal
to cure all diseases what we wanted to
do
it there are people there are serious
people who think
that all you know wireheading ourselves
is hooking us up to
dopamine just drips just like
supercharging our pleasure syndrome
is great that is exactly what we should
do they genuinely think that wireheading
is just morally correct that that is the
morally correct thing to do
and we should try to achieve that
possibility and i've read their essays
they make a good point like i i see
where they're coming from
but on that social media for example is
the ultimate example of wireheading
and why are heading for the ultimate
okay
let's hold this hold the thought for a
second wireheading philosophically
that's the cul-de-sac isn't it
when if you go and meditate or you study
mindfulness a lot of it is
a lot of it is teaching you to not be a
slave to your senses
and in a sense maybe that's implying
that we should be exploring the human
condition as much as possible
and getting stuck in these local
minimums or wireheading
where our senses get deranged
surely that's the worst possible thing
for human flourishing
potentially but there's perfectly
coherent arguments that say otherwise
it's perfectly coherent argument like
saying pleasure is good so more pleasure
is good don't make this more complicated
than it has to be
like that is a perfectly coherent view
whether or not you agree with it it's
perfectly coherent
but it's not good i actually don't
think yeah that's one theory you can set
up ethical axioms that come to that
conclusion
but it's very hard to say like there's
other people that say the only true
you know goodness is serving god or
being
chaste and serving the church or
whatever and these deontological
thoughts
and they say everything is not that is
automatically evil and
it's very there's this concept of moral
uncertainty which is very popular
recently with like rationalists and
ineffective altruist type people there's
a book that i have to read by toby orr
that i really have to read that's
all about this and this idea of we have
these different moral theories
that seem completely incompatible is
there some way we can
determine which one is objectively
better or that we can combine them in
some way
it's very unclear if we are a super
advanced super civilization you know we
have ais we
we've become enlightened post humans or
whatever and we come to some primitive
village
somewhere in whatever and they say hey
hello we offer you immortality and
internal happiness and they say no
should we accept that or should we say
no we know better than you this is a
really difficult problem
yes it is it's a very hard problem it
even the intuition is with the wire
heading we have a dopamine system in our
brain
yeah we can just rationalize that
clearly it's it
there's bugs in the software or in the
hardware yeah the only reason we have
that is because it encourages
behaviors that will allow us to
reproduce and and to
succeed biologically yeah so taking that
to the extreme and just getting into an
infinite loop on dopamine
we know that's not a good thing as i say
you say you know that but based on what
intuition can you formalize that
like there are ways of formalizing that
but you have to commit to it
and i can promise you that whichever
formalization you use will have some
strange education
this is just a thing that you get used
to in moral philosophy
is that you say oh this theory says to
be nice to people
so this should be a good theory and then
some other philosophy press says
oh so if you follow your theory actually
you have to murder millions of innocent
babies
and it's like this happens all the time
yeah there are some interesting thought
experiments in philosophy as well
there's the experience machine which is
uh yeah imagine you could go into a
virtual world where everything was
perfect would you want to leave
or another one is um what if we all died
in our sleep tonight and no one suffered
would that would that be for the good or
for the bad yeah
those very much depend on your ethical
framework of philosophy you said can you
and
there's an interesting thing of like
epistemology versus knowledge is
epistemology is interesting because it's
like your theory of how you gain
knowledge
and there are obviously epistemologies
that are better than other
epistemologies
like there are epistemologies that will
assuming we
the reality is real assuming reality has
properties that we think it does
then there are some epistemologies that
will lead to systematically better
results like bayesian epistemologies
like the provably
optimal way of doing it or whatever and
it's interesting okay if you come to a
belief if you randomly if you roll up
a large dice and a bunch of dice and
that's your belief about the universe it
happens to be correct
what does that mean is there a
difference between
you coming to a conclusion through this
is something i tried to form
i tried to express in the last uh talk
which i'm not super happy about how i
expressed it
i could express much better is the
question is that like a lot of people
talk about intelligence like oh it has
to have reasoning oh it has to do this
it has to do that so that
the question is if you have an algorithm
let's say you have a
cellular automata yeah like just random
black and white cells that produce some
kind of pattern
and we happen to find one super bizarre
weird cellular automata that just
happens to output only correct truths
about the universe all the time
not because it has any connection to the
universe it never perceives the universe
it just
happens to have that mathematical
structure is it knowledge
is this intelligent that's why these
formalizations of intelligence that
don't
aren't based on mr mentality on that
there's a difference though so
one instance is that just through random
chance
it happened to make all of the right
decisions so far and another instance is
a bit like gpt3 there is actually some
mathematical deterministic structure
there so we can
rely on it somewhat to keep getting it
right yeah here's the problem
randomness isn't really a property that
things can have it's like
the randomness is actually a complicated
philosophical theory it's like how would
you know that something is random
and not just the output of some very
complex
system like where is the difference
between
a large number and a random large number
like they're both the same number the
randomness is not a property of the of
an
algorithm or an output it is in our
world
but that's just for example you might
have a random number and an error that
takes in like input from
atmospheric noise or whatever but that's
not actually random
if we had perfect information about
every state of every particle
like okay we're seeing quantum physics
aside for the moment here that's a
that's another argument you could talk
about
is quantum physics actually random or
not that it might be it might not be
but it's randomness is not a property
that you know something necessarily so
for example like
you cannot predict any given digit of pi
as far as we know pi is completely
random but it's still
determined we still it's still going to
be the same number every time
like you don't know what the 10 to the
power of 100 digit of pi is
but if you ever find out it's always
going to be the same one it's
is that random does that count it's
random is a philosophically actually
difficult property that isn't
fundamental is
at least in my view of the thing
randomness is not a fundamental property
of an algorithm
it's just something that from our
perspective looks random so
in computational complexity theory they
often talk about pseudorandomness in
this way that of
the hardness it is to prove whether or
not it's random
so you can make algorithms that output
numbers
that you can't prove are not random in a
certain amount of time
you could say to prove that this number
is not random you need exponential time
for example
but you can't really make an algorithm
that
you know that if the numbers come from
somewhere there's some algorithm produce
these numbers
so if you just search through all
possible programs
which would take infinite time but in
theory could be done
then at some point you would find the
program that produced this number or
some other of the infinite programs that
produced that exact number
is it still random amazing connolly
thank you very much
indeed pleasure brilliant so on the
subject of max welling
he published this article in response to
rich
sutton's the bitter lesson so in the
bitter lesson he basically said that we
just need
infinite amounts of compute and that
will basically pave the way to
artificial general intelligence
now max welding came along and he said
okay wait a minute rich
you're forgetting data as one of the key
ingredients
because in alphago you can basically
generate the data yourself it's such a
narrowly defined
domain you can generate as much as you
want but anyway he said that there are
all of these contrasting views in the
machine learning world where people
have huge arguments about one of which
wallet suburb was representing last
week which is do we have compute driven
ai or do we have human knowledge based
ai
another one is model driven or data
driven ai another one is
symbolic ai or statistical ai another
one is white box ai or black box ai and
another one is generative or
discriminative ai
so those are the big inflection points
in our community
are people ever going to learn that
there are no silver bullets and
it's it's not a zero or one world that
the truth is always somewhere in the
span
between extremes i don't know how many
times we have to learn this but did you
just say the truth is
always in the span between extremes
that's a good question so am i
absolutely sure there are no absolutes
yes there's only one absolute the
universe is infinite is the only truth
you sure about that convince me convince
you that the universe is infinite
yep that would take a while man we'd
have to talk about omega
and the size of the observable universe
and how flat it is and i don't literally
know that it's infinite i
i know that i think the latest estimate
is that however big it is
it's at least 100 times larger than the
observable universe
something like that but does it have a
manifold
i can only tell you that it's if it's
not flat
that it's whatever this number is i
don't recall but it's
many times 20 to 100 times larger than
the
observable universe and
the most parsimonious explanation is
that it just so happens to be
exactly flat and therefore it's infinite
so it's flat and isotropic
and that implies that it's infinite no
it doesn't because you
yeah it does because any flat geometry
that is uh you know isotropic just means
it's the same in all direction
that's right and for example the only
flat
unbounded manifolds right
our euclidean space and
the what is it the four taurus because
we do have three plus one dimensions
space time and so it's the and the four
taurus
is not isotropic so you can show that if
we are in a
for taurus i think is the correct term
for it then there are directions in
space that have
an isotropy and and that's not detected
so you think we are not on a curved
space
the four taurus isn't curved it's just
unbounded
so it doesn't have curvature but it does
have anisotropies because if you think
about
just think about the simple case of a
normal torus which you can view as say a
square
where you identify the left edge with
the right edge and
the bottom edge with the top edge like
that can exist
without intrinsic curvature or without
without a topological curvature
and yet on the other hand if you travel
you can travel in different directions
and it'll take you longer or less time
to get back to
the point where you start if you just go
straight up in the y dimension or
whatever you wrap around one time and
you get back to where you were
whereas if you go diagonally you can
thread around multiple times around the
torus before you get back to where you
started
and so that introduces anisotropies
in the in the observation so it's either
something like a
very slightly curved say four sphere
or it's flat and if it's flat i think
the only two possible flat
unbounded geometries are infinite
euclidean space
and finite for taurus
i guess it's really a three taurus for
space and then and then time
the time dimension you know treat in a
special way how did we get to the
universe
you asked me if i was something like am
i sure there's
100 that the the truth is 100 of the
time
between extremes so i see that was an
example for
maybe sometimes the truth is
at the extreme yeah are you absolutely
sure there are no absolutes
and do you tolerate intolerance for
example
are you tolerant of intolerance remember
to subscribe
we'll be back next week see you folks
you announced we have a very special
guest today
you remember scheduling confusion yeah
let's just not hype that anymore our
guest was so special they faded away
into into nothing have you seen the oh
what's this
rock this rockumentary parody spinal tap
this is fine have you seen that what
happened to your last drummer
he just disappeared exactly
yeah he became so specialized he just
became impossibly thin
all right catch you later absolute
pleasure amazing cheers guys take care
bye-bye
bye-bye i really hope you've enjoyed the
show today remember to tune in next week
we have two amazing guests coming on the
show and i can't wait for you to see
the result of that remember to like
comment
and subscribe we love reading your
comments and we'll see you back
next week