Timelines for Transformative AI and Language Model Alignment | Ajeya Cotra

hi everyone welcome my name is felipe
and welcome to our fireside chat with
ajaya kotra on timelines for
transformative ai and language model
alignments ajay katra is a senior
research analyst at open philanthropy
she recently published a report
estimating when transformative ai may be
developed using biological anchors which
is an input
into how much capital should be
allocated to the risks from advanced ai
focus focus area relative to other areas
she also collaborated extensively in
writing the alignment research center's
recent report eliciting latent knowledge
and published a piece advocating for
aligning narrowly superhuman models
so let's get started welcome jaya
thank you felipe
great to be here
so to start us off um have your views
changed at all since writing the the
post on on nearly unaligning narrowly
superhuman models
yeah um i think that uh
since writing that post i've been
thinking
more deeply about various different
aspects of alignment and i would say i
kind of have more
specific views on like which kinds of uh
empirical alignment
uh projects are good and why um and i
think i would broadly still um
stand by the aligning narrowly
superhuman models point of kind of look
for places where
ai systems are better than humans
already at certain things and and try to
find ways to
um get them to be as helpful as possible
but i think there's kind of
i've thought a bit more about things
like interpretability
um or just kind of
particular projects that don't look like
that that might still be
helpful um and maybe kind of gradations
and how helpful different versions of
aligning super nearly superhuman models
might be
[Music]
and when you say useful you mean like uh
in terms of like usefulness for
alignment research or usefulness for for
users uh usefulness for kind of
preventing
super intelligent style scenarios in
which
um
ai systems that are
trying really hard to maximize something
that humans
don't want
ending up kind of getting uh all of the
power in society
[Music]
and maybe on that point how do you see
um
research on on this narrow topic
connecting with the broader broader or
like core
alignment
problem
um research on the aligning nearly
superhuman models
um yeah so
basically the concept here is that in
the future we're going to have ai
systems that are smarter than humans
better than humans in a wide variety of
domains
and that brings
particular challenges in controlling
them and making sure that they're not
going to do something that we don't want
to do
because once they're that powerful if
they're
trying really hard to accomplish
something that is not quite what we
wanted um like trying to get the literal
reward number um in their computer to be
a really high number that could lead to
really destructive kind of power seeking
actions such as trying to gain physical
control of their computer to
you know set the value to a high reward
for example um so like a
chief challenge of trying to do research
now to prevent
this future outcome is that systems now
aren't powerful enough to
um
take really scary open-ended actions in
the course of achieving their goals so
you know uh starcraft playing ai
isn't smart enough now to try to
uh bribe its opponent into throwing the
game in order to get it to win um it's
kind of got this limited action set
which involves like doing things in the
world of starcraft
and so even though it's trying really
hard to win at starcraft that doesn't
lead to kind of
scary open-ended power-seeking or
deceptive actions
um
and so one potential
patch or way to try to study something
useful anyway today
would
be to look for certain specific domains
where ai systems do know something that
humans don't so for example an ai system
might
know more physics than a
certain group of humans or something um
and trying to get those ai systems to
be helpful to the humans even when if
the humans were kind of naively looking
at
their outputs the humans would be kind
of rewarding them for the wrong thing um
so a human that has misconceptions about
like tough physics problems um it's it's
sort of not obvious how to get those
humans to
give useful feedback to these models
um without incentivizing the models to
just deceive the humans
um and that is potentially usefully
analogous to this problem where in the
future
models might be kind of open-endedly
trying to optimize for reward or
whatever else they want
and then that might
kind of
lead them to
try and
seek physical control or seek power over
humans or try to remove human control
over their
training environment or their rewards
thanks um and one question from the chat
um do you know anyone addressing or
working on the issue of uh perhaps
language models only knowing how words
are used and not having sort of symbolic
knowledge about uh
how they what these words actually mean
um
there are definitely people who think
about that question um my sis uh best
guess is that um
in the future as language models get
larger and larger and are trained on
higher and higher quality data sets um
we'll see more and more coherence
of the kind that this question is
gesturing at emerging so
you know language models of the past
were were very much just kind of
repeating shallow statistical patterns
and words um language models today are
capable of kind of sometimes doing
things that look like reasoning so you
know most recently language models did
pretty well on kind of grade school math
word problems um and i think what that
trend will continue and language models
a few years from now will be a lot
better at
kind of
coherence and logical or symbolic
reasoning without
a particular algorithmic intervention
that's like specifically targeting that
although like just a lot of
interventions
or a lot of innovations will just kind
of make it
easier to and better to train these
models
to be useful
now pivoting to your report on on using
biological anchors to estimate when
transformative ai may be developed um
what do you make of the recent critique
by eliezer yorkowski
on on this report
uh yeah i mostly agree with holden's
response to that critique um
eliezer is sort of
making the assumption that the report
assumes that the only way to get
transformative ai is by training these
large models
um but
the sort of real thing that i think is
needed to get that analysis off the
ground is to expect that that's a way to
train these models um so
thinking about that question can still
provide a rough upper bound
um and kind of the reason that
i
treat these this kind of rough upper
bound as more like a median is that um i
look at one input which is compute
and i i didn't specifically think about
when the data or environments might be
ready um and
i
generally feel like sort of
extraordinary and extreme claims
um face a certain burden of proof
um
so i think eliezer's perspective here is
something like
that's just like extremely uninformative
um and i disagree with that
um though i do agree with him that like
it isn't uh
like the way the report is structured
it's not automatically going to knock
down anyone's particular
uh inside view
sense of how you might build a gi that
might include elements other than deep
learning um
you know i'm generally somewhat
skeptical of like inside views on
um
how we might build agi that
result in the belief that agi is really
soon
but that's kind of a separate thing from
biological anchors and each of those
arguments would kind of need to be
examined in turn
and i think eleazar has something in
mind that for infohazard reasons he's
not sharing um
but
i think the report is kind of more
responsive to someone who is either like
i have no idea what to think about this
question or
believes that agi is definitely hundreds
of years away and is like not something
that could possibly happen soon um so i
think the report is kind of a more
powerful response to that kind of
perspective
and then a similar question um
to the the views changing
have you been surprised by any recent
developments have your timelines changed
significantly since since writing that
report
not significantly
i think
a mildly surprising recent development
was the deepmind language model
models gopher and retro which were
published
a few months ago but actually had been
apparently developed
more than a year ago
so i was sort of wondering why we didn't
have kind of bigger and better language
models already and it was kind of
interesting to learn that this group had
kind of had them and been been sort of
sitting on them for a while
um but but overall but my reasoning is
like pretty similar it's kind of
acquired a lot of
texture
and some specific beliefs have changed
but um
no dramatic shift since then
and in the model one of the inputs is
sort of algorithmic progress and um from
the chat we have a question uh how
linear is algorithmic progress what
probability would you put on a
discontinuous jump in the future pre-agi
yeah so um i think the kind of
principled thing that um
i would have tried to do if i had more
time on that report would be to um
instead of
projecting algorithmic progress
as one point estimate um with uh that's
changing over time having a distribution
that's changing over time um rather than
saying you know the amount of compute
required to train transformative ai uh
halves every two or three years um it
would be saying there's a you know x
percent probability that it reduces by
10 and there's a live percent
probability that it reduces by 10x and
so on um and just
prima fascia i would expect that what
that looks like is that there's kind of
a distribution of innovations
um there are a bunch of people trying a
bunch of different things
um writing papers in um a given year um
and
you know the
the distribution
would probably look like a lot of papers
help a little bit
um and some papers help a lot um and
some papers help so much that they only
come along
once every 10 years or once every 20
years um the more people
are working in the field already trying
to like
uh claim the low-hanging fruit the less
likely it is that
we're gonna see a giant discontinuous
jump
um so that's kind of my my general
background picture so i definitely think
that um a better version of the model
wouldn't would include that uncertainty
um and you know my my basic answer to
what's the probability of a big jump is
uh it really depends on how big that
jump is um and i think my probability
over time will be going down um because
more people will be entering the field
and um we'll be trying harder so there's
more of an opportunity to for a big
surprise like you know next year than 10
years from now when the field is 10
times as big
makes sense
[Music]
and another question uh from from the
chat um
how much do you think compute is is
really the factor that will be the sort
of determining
um factor in when we get to automate
resistance developments and having
superhuman models um
and how how much uncertainty you have
for that versus it being sort of data or
environments or
algorithms um and how do you build that
into your mod
um
yeah i think it's a compute is pretty
likely to play a large role historically
speaking
[Music]
you know progress in the field has
tracked pretty well with the amount of
compute in the field compute kind of
helps in two different ways
one is that
in a direct way you're able to train
larger models but another is that it
makes experimentation a lot cheaper um
so you can try more things if uh running
a b sized model is just a trivial thing
versus if it's like a very tough thing
to do
um so i expect compute to be a pretty
big deal i also expect data to be a big
deal um but i think we
are
as a civilization we haven't tried
nearly as hard yet
creating
high quality data sets for these models
we've mostly just been kind of passive
we've been like scraping from the
internet very occasionally we're like
just dipping into using human feedback i
think there's a lot more we could do in
that direction in terms of like paying
humans to produce high quality data and
high quality simulations to train these
models on so i i expect that to be less
of a bottleneck because i think there's
like much more headroom and much more
low hanging fruit whereas
computers computer hardware is something
that like as a civilization we've we've
put in a huge amount of effort into for
many decades um and it's kind of like
one of the the largest scale
most complicated industrial projects
um in our entire society to just get it
a little bit better um and so i expect
that to be more of a bottleneck just
because we've like kind of rung out more
of the
the like easy wins there and we we've
done that a lot less for data and
environments though i definitely think
it'll still be a lift and it could
easily be the case especially if the
compute requirements are pretty low it
could easily be the case that we're
bottlenecked for multiple years on
creating the right data sets in the
right environments
[Music]
and now pivoting more to
the alignment side of things
what are some things you might observe
in the short term which would reduce
your uncertainty about the broader
problem of alignments
i think that's a really tough question
um there's kind of two categories here
the easiest changes i could see
would be just to see ai labs and people
in industry taking the problem seriously
and seeming to kind of understand
the same version of the problem that i
understand it to be so a lot of people
working on ml safety right now are
mostly working on
making existing systems more robust more
reliable less likely to say offensive
things um
easier to use
all of which are good um but don't
particularly track strongly with this uh
this kind of problem that will emerge
uh or that i think is likely to emerge
when we have
kind of powerful large open-ended
systems
um and
you know specifically this thing where
um no matter what they want it might be
pretty helpful for them to
take control from humans once that
becomes easy enough to do
so not a lot of people
are particularly worried about this
problem and i think that
in itself is a source of a lot of the
danger in my mind
um if kind of there was a societal
consensus that well the default easiest
way of making powerful ai
runs like an unacceptably high risk of
this outcome so obviously we need to
figure out how to do something else then
my personal estimate of risk would drop
a lot um because that's like the the
first and maybe most important step to
solving it so it might not be
particularly difficult to solve this
problem but it will require i think at
least some
mindfulness coordination
and like directed effort toward it um
that we don't
particularly have right now um on the
technical side i think it's like
much
less clear and much dicier
what kinds of empirical evidence would
make me feel better um i think i'm more
likely to update based on kind of
theoretical arguments
at this point than
empirical evidence so
you know an obvious thing would be if
the alignment research center which is
kind of trying to solve alignment on
paper
says they
found a solution to alignment on paper
or or something that um
solves uh an important sub-problem of
alignment then that would definitely
make me feel better
could you say a bit more about the sort
of like experts and
leaders in tech companies not engaging
with the same sort of problem
you are can you say a bit more about why
you think that's the case has it sort of
been not engaging enough with the
arguments not finding them convincing or
sort of dismissing it out of hands
i think it's a complicated situation um
i think
the worry
that i kind of take most seriously in
this space is pretty sci-fi you know
it's quite similar to like the
terminator or ai takeover scenarios so
people generally
um
are skeptical that something so
extreme and weird um is really where
we're headed and i think that's like
kind of
that's kind of the the starting point
for a lot of people um and then from
there people have tried to engage with
them made cert
certain arguments and they've generally
been kind of unpersuaded in my
experience
um and
in my experience also don't kind of
seem to have as detailed and
understanding of the case for this as i
would like
um
so you know one of the things i'm hoping
to work on over the next few months
having more conversations like trying to
articulate what i think is the simplest
and strongest version of the case that
uh
you know this this whole terminator
thing might might actually be real and
might actually happen
and then uh a question from the chat um
so so as as background there's been
people that that think um they you might
get closer to agi or transfer to ai than
we think by just scaling up language
models um and so in that vein how
concerned are you about deceptive
alignments or misaligned mesa optimizers
in language models specifically as
opposed to transformative ai and other
systems of ai as well
yeah um i think that uh
for sufficiently powerful language
models even if they're just doing
prediction
it's pretty plausible that being
a consequentialist and like kind of
thinking explicitly about what thoughts
you should think and what avenues you
should explore what hypotheses you
should explore in order to get the best
prediction will turn out to be
um will turn out to be useful so you
know when you're when you're a very very
dumb predictor um then picking up on
like really crude statistical patterns
might be the lowest hanging fruit and
when you're really when you're pretty
good like gpt3 then picking up on like
kind of grammatical rules and things
like that might be the lowest hanging
fruit and maybe when you're very very
good um
and you've done all of those things then
then being kind of internally
consequentialist about like exactly what
avenues you explore and don't explore
might be the lowest hanging fruit in
terms of becoming a better predictor so
um
you know even pure language predictors
might well become
internally consequentialist and that
might mean they have goals that aren't
quite
uh what we would want and uh pursuing
those goals really hard might lead to
uh sort of power seeking or ai takeover
scenario um with that said i think i'm
more worried about language models that
have been further fine-tuned with
reinforcement learning than language
models that are pure predictors just
because i think that it's pretty
unlikely that we won't do reinforcement
learning to some degree and
reinforcement learning kind of
um is a much harder nudge i would guess
toward being a consequentialist that's
trying to pursue some goals than peer
prediction even though peer prediction
could do it as well
i know there's another chat question
about the broader awareness
topic we were just talking about um
how concerned are you about the the
potential trade-offs between broad
awareness being useful but potentially
concerning if it causes for example
governments in a way that's concerning
to be more involved
our malicious actors that may have
otherwise been very
focused on
on ai
yeah i think it's definitely concerning
and a tough line to walk and concerns
about this sort of thing are part of why
we
um didn't try and
kind of uh market the timelines report
very much
um i think it's probably possible i
think there's probably something you
could say that would be very positive
along the lines of you know the default
way that we can maybe make powerful ai
would be really scary and here's why
with more of the emphasis on it would be
really scary and um
less of the emphasis on and we could do
it for this amount of money
like this soon so here's like a general
template and once we have enough
resources who knows what those resources
might be um we could use something like
this to train a powerful ai and that
thing seems really scary um i think is
is a good message that i think like kind
of
has a good cost benefit analysis if
you've done it thoughtfully
um i also think
do this type of research to
[Music]
you know make it less likely we have a
robot apocalypse is a good message um
that i think uh has you know more more
benefits than it costs and really the
the dicier message which is uh
it's tough because i think you do need
to say some of this to get people
interested the dicer message is like no
this really could happen soon it really
wouldn't
necessarily be that much money and kind
of getting into the details of that so
our kind of approach has been
downplay that um you know have the
analysis there to point people to but
don't draw a ton of attention to it and
be much louder about the the potential
worries um
and um and most loud about potential
solutions
i know jumping back a bit um you
mentioned how future future models may
be um acquired sort of like
internal be like internal
consequentialists um can you say a bit
more about that and how can we be sure
for example that gbt3 does not have this
property
uh i think it's going to be very tough
to tell um because it's kind of a
property of what's going on inside the
model's brain and we don't have good
brain reading for models so we have
people who are trying to do
interpretability and mechanistic
transparency stuff like chris ola's team
at anthropic but um
you know that whole
science hasn't gotten very far um
and so i
think that it would be pretty tough to
devise tests of it for existing models
what i kind of mean
in smarter models is kind of something
that you might be able to access just
just introspectively humans often make
plans make goals um and also make plans
about what they should think about next
um you know what uh what aspects of a
problem they're facing like deserve more
cognitive attention and so that is the
kind of
move that i think might be really
valuable for
just generically all sorts of future
models even if they're just predicting
things it might make sense to be
consequentialists about their thoughts
in this way
[Music]
and you mentioned before about being
being
one thing that might change your mind is
sort of on paper results about
alignments um how optimistic are you
that that progress of this sort is
possible if say
um you knew that that um chance
was like
more than 40 years out or something like
this could you could we still make
progress even though we're not anywhere
close to these sorts of capabilities
i definitely think the case becomes
weaker um i still think there's a good
enough chance that even if
transformative ai is 40 years away
the basic principles
uh will still involve you know having
some sort of scoring function
and some sort of search function that
like finds a model that finds a program
that does well on your scoring function
um and i think even starting from that
very basic
assumption
which doesn't necessarily imagine neural
networks or gradient descent you could
kind of replace neural networks with
something else and replace gradient
descent with something else
i think there's still interesting
theoretical work to be done so i would
still be interested in in
funding that sort of thing but it would
definitely seem less urgent to me
[Music]
and then uh jumping back uh again a bit
to
broader awareness um
how much
um
how much of these ideas do you need need
to be um taken in by the people making
decisions is it okay is um
uh maybe this could be the last question
because we're running out of time um
does it need to be the leaders who are
aware of these ideas or is it sufficient
if there's a few people on the team who
are sort of concerned about this it can
bring up
um some problems
um i mean i definitely think that kind
of the more people
understand these issues deeply the
better um you know across
ai labs governments and so on and uh you
know deeply understand not just that
this powerful technology might be
possible but also that it might be very
very risky and maybe we shouldn't build
it
all right well thank you so much it
looks like that's all the time we have
um thank you so much to jaya for a
wonderful session and for your time with
us today um as always we invite everyone
attending to swap card to find fellow
conference attendees share your
interests and set up meetings
um if you need any assistance please
visit our siri help desk on swapcart
thank you everyone for coming thank you
[Music]
[Music]
so
[Music]
uh


Stuart Russel on AI and the Midas Touch problem #shorts

so
ai in the standard model
which has been
pretty much enforced since the beginning
of the field is that machines are
intelligent to the extent that their
actions can be expected to achieve their
objectives
so those objectives could be
logical goals for problem solving and
planning systems they could be
constraints for
csp solvers they could be
reward functions for mdp solvers or
reinforcement learning algorithms and so
on
or less functions for supervised
learning so
i would argue that the standard model is
flawed as a methodology
and instead we want a slightly different
definition we want machines that are
actually beneficial
to us
not to themselves so to speak and so um
we want it to be the case that their
actions can be expected to achieve our
objectives the objectives that are
actually in us
and not
in them


AI Alignment and Our Momentous Imperative to Get It Right by Olle Häggström

[Applause]
i am
very very glad to be here uh thank you
for the kind invitation to speak to this
audience
and and for the kind introduction let me
see if i can find my slides very good
okay so
uh
i think some of you
uh saw this
paper that came out
last month
on
[Music]
artificial intelligence for drug
discovery
this group
had a system
for
searching the space of molecules
and these molecules were supposed to
have some particular
effect inhibiting certain processes in
the body at the same time as being
generally non-toxic
and what they thought of and this was
kind of an afterthought was that
okay but what happens if we change this
one parameter from plus one to minus one
about the the value of toxicity what
would this system do and it turned out
that this system was very very efficient
in discovering uh molecules for
nerve gases
uh so it rediscovered
uh one of the worst nerve gases there is
and many other molecules where they
didn't implement them chemically but but
it seems that some of them are very very
dangerous so this points at the
dual use of many ai technologies and the
risks involved and we need to think
about these
um
[Music]
the association for computing machinery
is the oldest computer science
professional organization i think it was
founded in 1947
and one of the biggest maybe the biggest
they
said this
in march
2018
or their group for future of uh
computing
so they talked about the current status
in the computing community to be framing
our research
in papers and and in in
applications for funding and so on
through rose-colored glasses
these are the normal lenses through
which we tend to view our work we talk
about
how good things will be and this is
terribly terribly one-sided and they
compare this to the
pharmaceutical industry where
nobody would think about just talking
about the good parts
of what the drugs do but
and ignoring totally uh side effects you
cannot do that
and they had this recommendation
that
the peer review process in computing
should do something similar peer
reviewers should require that papers and
proposals rigorously consider all
reasonable broader impacts
both positive and negative this is kind
of i think this is ethically
straightforward to argue for we all have
have a responsibility for the
consequences of our actions and
uh we need to think about what these
consequences are
and
act accordingly
so
at chalmers when the chalmers ai
research center was launched one of the
first things
we did
was to formulate an ethical policy which
describes some of these ideas so so here
here's one passage an overarching
principle is that ai systems whose risk
of causing harm is not clearly
outweighed by their beneficial effects
should not be billed or disseminated
when estimating benefit versus harm it's
not always sufficient to consider the
problem from the viewpoints of
developers
owners and users of an ai system in many
cases there's a need to consider also
further stakeholders third parties
affected by the use as well if as
effects on the environment this
fundamental principle should never be
allowed to be overridden by commercial
military or other considerations
i'm not saying this is easy it certainly
is not but we always
need to try to think
carefully about these issues and then do
the right thing
and one of the things this requires is
to have
a broader picture
of what is going on and i'm going to
talk to you here about one part of this
broader picture
the part that i think in the long run
is the most crucial
and most influential and i when i talk
about the long run
i'm not just talking about the next 100
years
but also the next
thousand years or millions of years or
even longer time perspectives how we
handle
ai
can influence
the whole of human history
from now and on
let's start with alan turing
he most of his talk was very uh
mathematical uh and technical but
towards the end of his tragically short
life he allowed himself some more
philosophical speculations about where
we were heading and there's this passage
from 1951 he said my contention is that
machines can be constructed which will
simulate the behavior of the human mind
very closely let us now assume for the
sake of argument that these machines are
a genuine possibility and look at the
consequences of constructing them it
seems probable that once the machine
thinking method had started it would not
take long to outstrip our feeble powers
there would be no question about the
machines dying and they would be able to
converse with each other to sharpen
their wits at some stage therefore we
should have to expect the machines to
take control
now from this observation if you buy
this
there's there's a
kind of obvious corollary namely that
what happens from then on
depends crucially on what the machines
are motivated to do
and this was
spelled out a decade later by another
great
20th century hero
norbert wiener
he said
if we use
to achieve our purposes a mechanical
agency with whose operations we cannot
effectively interfere once we have
started it then we had better be quite
sure that the purpose put into the
machine is the purpose which we really
desire and not merely a colorful
imitation of it
this is very important message but it
was
almost entirely ignored by the academic
and research communities for the next
half century and only now are we
beginning to see
the emergence of the research areas of
ai safety and ai alignment
part of which
whose goal is precisely to make sure
that the goal we put into the machine
or the purposes
are those that we really desire and not
something that just superficially looks
like it if we succeed in this
the
future can be
flourishing for humanity on
unimaginably
great levels and if we fail
then
we have failed
entirely
um
so to think about
failure modes
um
there have been suggestions that
maybe the first real artificial general
intelligence
reaching super intelligent levels has
imagine a scenario where it has the task
the goal of computing as many decimals
as possible of the number
pi and this can i mean for an ordinary
uh software system
this is
of course not dangerous but for a system
which is super intelligent this can go
arbitrarily wrong
we can imagine a scenario where it puts
unlimited efforts into turning
as
much matter as it can including our
entire planet and ourselves and so on
into more and more hardware allowing it
to more efficiently compute more and
more decimals of pi
there's a similar example
that is sometimes uh discussed
where the task is to produce as many
paper clips as possible with a similar
outcome we don't want this uh to happen
these are very particular examples there
is one more i want to mention which is
slightly more generic
and which is that
when you construct
an ai system
with a goal
you will often have
some um part of the machine's memory
system that that uh
keeps track of how well this goal is
received a number
telling how things how good things are
going for the ai
and it's very very difficult to
uh distinguish when pro programming the
machine the
the goal that we really want
and the
proximate goal of
just maximizing this number
and if the machine
realizes that it can maximize this
number through other means
this is called wire heading and it also
happens in humans in in in various ways
if the machine figures this out
something similar to to the pi
calculator might happen it just captures
the whole world to construct more and
more hardware to be able to represent
more and more nines in this ever larger
number of how
well things are going for the ai they
will be going super well for the ai not
so well for us we should avoid this kind
of scenarios and one of the insights
in this emerging field is that this is
not as easy as it sounds it's probably
not undoable it's worth working on but
it's certainly not
trivial
this is controversial there are
very different opinions about this kind
of work
uh in in in the ai community here is
leading ai researcher andrew eng he has
a very catchy quote from a few years ago
where he says there could be a race of
killer or what's in the far future but i
don't work on not turning ai evil today
for the same reason i don't worry about
the problem of overpopulation on the
planet mars
okay that's catchy
but but as as another leading ai
researcher stewart russell has responded
this is somewhat
irresponsible and he says within the ai
community a kind of denialism is
emerging even going as far as denying
the possibility of success in achieving
the long-term goals of ai it's as if a
bus driver with all of humanity as
passengers said yes i am driving as hard
as i can towards a cliff but trust me we
will run out of gas before we get there
should we
rely on this
i think not i cannot tell you when
transformative ai breakthroughs will
happen
we have to
have epistemic humility about this
but that does not mean that
this is certainly going to be into the
far future there are very different
opinions about this when you make
surveys and the one reference i would
like to point you to if you really want
to dig deeper
into
this question is ajaya kotras 2020
report on forecasting transformative ai
with uh biological anchors
uh much of this is conjectural and and
and speculative but she does as best as
she can
biological anchors she's very agnostic
about what kinds of biological processes
are the right
metaphor for understanding this is it
the computing power of the human brain
is it the amount of
information that is required to bring up
a newborn baby to age 20
all the stuff the child learns along
then or is it the entire amount of
information in the in the full
biological process on the planet earth
and and she has other candidates and
each of these gives an uncertain answer
to how much computing power
is necessary with
2020 level algorithms
for
creating super intelligence and all this
uh gives um so so so this is a plot on a
log scale on on how much it's a
probability distribution her best
attempt at that for how much computing
you need
this big part of the distribution spans
over
20 or 25 orders of magnitude is very
very uncertain and then she goes on to
translate this into
a timetable
by
invoking
trent technological trends and economic
trends in how much companies are willing
to invest she compares it to the
manhattan project and the apollo project
basically the biggest technological
projects we have seen so far
and
this lands in in a very spread out
probability distribution for when we
might get transformative artificial
intelligence so we're working against a
very uncertain
timetable
and personally i would say i would be
even more agnostic and spread out the
distribution even more than this but but
you you may notice that the the the mode
or the median is somewhere around
20 20
uh
45 24 to 2045 something like that but
it's very very spread out
the key reason why we need to work on on
ai safety
right now
is not that we expect it to happen next
year or in two years or five years this
could happen not super likely but the
real reason is that
we need all the time we can we can get
for uh solving this problem so no
procrastination please so what makes
people skeptical in the light of this
kind of evidence
uh people like andrew eng and and i can
mostly just speculate about this that
but but i think that a key intuition
which drive
many agi skeptics is the so-called
common sense arguments
we have all seen videos
of robots doing silly things tripping
over their toes and and so on and we say
to ourselves oh
these these machines are are so stupid
they lack common sense so um
transformative ai
real breakthrough will be very far away
and in the fall of 2020 uh there was
this particular example where they had
programmed
an ai to steer a tv camera during a
football game to track the ball and this
was supposed to make good tv production
for the viewers but the the ai made the
mistake of instead tracking the bald
head of one of the lines men
a human would never ever make this
mistake ai's are so stupid agi is far
far away because of this lack of common
sense but imagine being an ai
pointing to humans
here here's a chess game between
uh an ai and human the poor human
playing black here exposes himself to a
devastating attack along the diagonal
down towards the king and it's such an
obviously non-common sensical view of
playing chess even though it's a human
grandmaster
from the ai's point of view that the ai
might say humans obviously lack common
sense the real situation here
is that
there are
things that humans are better at
and there are things that more and more
things as time progresses that ais are
better at
and and and we tend to put this label
common sense on precisely those things
where humans
are
still better than ai's and then implicit
in this argument is that before anything
drastic can happen
uh ai needs to pick up all these common
sense
competencies i think that's the wrong
way to think about this i think the
right way to think about it is more
along the lines not of when will ai
exceed us in all cognitive domains but
rather when will ai exceed us in enough
domains to be better than us at taking
control
of the world
and we we as i said
we really
don't know
so another
counter-argument that you often hear is
wouldn't the super-intelligent machine
understand
the wrongness
of killing us and this is an argument
that pops up every now and then last
year
philosophers uh vincent miller and
michael cannon uh had a paper where they
expressed this intuition and turned it
into kind of
this is a scientific paper
uh
and
basically they talk about this notion of
artificial general intelligence and they
suggest that if if an ai is generally
intelligent
then
that includes moral intelligence and
that means that the machine would
understand the wrongness
of killing
um
we have seen in very recent history that
not all humans understand the wrongness
of killing and it's not just that
but it's also
i mean every computer programmer
knows that machines do not automatically
do what we intend them to do and they
have do not have the property of trying
on their own to figure out what it is
that we want they do what they are
programmed to do not what the programmer
intended
and for for from the point of view of
values that we all
share here we love things like human
flourishing
and
uh let's say biodiversity maximization
and so on
uh it's i explained this in in a
response paper that i wrote um a few
months uh later
uh ai orthogonality and the miller uh
canon
uh
instrumental versus general intelligence
distinction
um
so so from our point of view it's it's
obvious that
uh
promoting human flourishing is the right
thing and that maximizing paper clip
production is the wrong thing but from
the point of view of the machine that
has the goal of
maximizing paper clip reduction
things look the other way around the
machine might think to itself okay human
flourishing that's something but but but
it's really stupid to prioritize that
because it doesn't lead to paperclip
production uh necessarily
so so so
to go a little bit deeper into this uh i
i think it's uh
i recommend these two papers and and you
can be the judges
uh
i i think that it's not entirely
impossible that there will be automatic
mechanisms steering us to a a
happy future maybe if moral realism is
true moral
uh
cognitivism and moral internal
internalism these are complicated
philosophical positions that need to
have they are wide open but if they have
precisely the right answers then maybe
things will work out on their own but i
think it's wrong to take this for
granted
okay
uh so going back to the quote by wiener
he suggested
that we need to make sure that the
purpose put into the machine is the
purpose which we really desire not
merely a colorful imitation this is
known today as a.i alignment
and we can
very roughly
and and and it doesn't really cover all
aspects but but most of them this can be
divided into theoretical top-down work
which dominated the field until a few
years ago
because i mean it's very hard to
experiment
on something that you don't already have
but with the deep learning revolution
and with
the great
progress for instance in natural
language processors there have opened up
the the possibilities of meaningful
experiments so we're beginning to see an
increasing amount of experimental bottom
up work but there's a limit to how much
you can do with experimentation because
i mean it's very very dangerous to to
let a super intelligent machine out in
the wild
uh and and and expect that you can stop
it uh if things can go wrong so so
there's this i mean we need both of
these aspects and i'll just finish by
mentioning some of the organizations and
some of the people
that have been doing these things
the top-down approach here
we have the machine intelligence
research institute in berkeley
uh where who
whose founder is elsie yukovsky
uh
sort of a leader in the ai safety field
there's also the future of humanity
institute at oxford with people like
nick bostrom and stuart armstrong
if you look at the bottom-up approach
very interestingly some of
uh the most important actors working
towards artificial general intelligence
are also interested in these safety
issues open ai
you know the gpt-2 and gpt3 products and
codecs and so on
uh they're doing work uh in this
direction
uh these are daniela amuday dario amuday
and chris ola
the they're
some of them were on this paper concrete
problems in safety from 2016 which
opened up very interesting ideas
connecting
um
down-to-earth systems like household
robots and so on and
showing that these are some of the
problems in making them safe is the same
kind of problems we need to solve
on this larger scale the other
organization i want to mention is deep
mind
where we have people like victoria
krakowner
and jeffrey irving uh also doing some of
the best
present-day cutting-edge work on ai
safety there are many others
uh let me just finish by
mentioning some go-to places for what to
read about this stuff nick bostrom's
super intelligence from 2014 is
extremely influential it's a wonderful
book
but it also
i mean
we have gone beyond
what uh bostrom talked about in 2014
so so so
it's it's
a good book for uh getting the early
ideas but uh uh
to to to know what's happening today we
have to go beyond this stuart russell's
human compatible from 2019
is
wonderful uh
brian christian
uh
he he talks about the alignment
alignment problem in his 2020 book from
a more journalistic perspective
and finally for those of you who read
swedish i want to mention my own 2021
book
uh
thank ande mariner
where i try to sketch
um some of the uh research directions um
that that we're working on in this field
and with that i thank you for your
attention
thank you
so when you're deep down in uh
debugging tensor dimensions in in pi
torch it's really hard to sort of see
where the direction where you're going
but it's important to that we all get to
know that
i i think it's unrealistic that everyone
should be thinking about this uh
at every second of their work but i
think that we can ask that everyone
every now and then
uh takes a step back and thinks about
the bigger picture and what it implies
for their work for sure and sort of we
forced you to do that today
we have one time for one question i
think yeah so can we get the mic over
here
um
hi um i think it was a very interesting
talk and thank you for that
we talked about super intelligent ais
and
machine learning or ai turning evil and
i think we are already seeing some
symptoms of it
for example certain chat box
says going super racist or biased
and it's because we are not living in a
perfect world so our
data set that is trained on is not
perfect and it is already biased do you
have any comments on it like
yeah
okay so so uh having appropriate data
sets
is certainly
uh
part
of this problem
and
and fixing it uh is is one part of what
we need to do
to get a solution but it's it's not a
total fix of the problem there are many
other things we need to do and at
present
we don't even know how to encode human
values in in machine language we we
don't even know i mean we don't even
agree with these values are so so so
these are examples of
different kinds of things but you're
certainly pointing to to one important
aspect and an aspect that is important
in down to earth applications today when
we have computer ai systems
uh judging who should be eligible for
financial support and even in in
criminal courts and so on to judge
people's tendency to relapse into crime
and so on so so certainly yes thank you
you


Ethan Caballero–Scale Is All You Need

A bunch of people at Google
said, yeah, we have language
models that are way bigger than GPT-3,
but we just don’t put them in papers.
The DeepMind language models papers,
they were a year old when they finally
put them out on arXiv or whatever.
When Ilya tweeted the consciousness tweet,
they were like, goddamn, GPT-4 must be crazy.
Like there’s a zillion VCs throwing money
at large language model startups right now.
At some point, the Beijing Academy of AI will
be like, look, we just trained a 10 to the 15
parameter model on all of YouTube and spent like
$40 billion doing it. And then at that point,
Jared Kaplan’s gonna be in the White House
press conference room will be like, look,
see these straight lines on log log
pots, we gotta do this in the USA now.
The Inside View. The Inside View. The Inside View.
Ethan, you’re a master’s degree
student at Mila in Montreal,
you have published papers on out of distribution,
generalization, and robustness generalization
accepted as presentations and spotlight
presentations at ICML and NeurIPS. You’ve recently
been thinking about scaling laws, both as an
organizer and speaker for the first neural scaling
laws workshop in Montreal. You’re currently
thinking about the monotonic scaling behaviors
for downstream and upstream task, like
in the GPT-3 paper, and most importantly,
people often introduce you as the edgiest person
at Mila on Twitter, and that’s the reason why
you’re here today. So thanks, Ethan, for coming
on the show and it’s a pleasure to have you.
Likewise.
Scaling Laws T-Shirts
You’re also well-known for
publicizing some sweatshirt mentioning
scale is all you need AGI is coming.
Yeah.
How did those sweatshirts appear?
Yeah, there was a guy named Jordi
Armengol-Estapé who interned at Mila,
and he got really into scaling laws, apparently
via me. And then he sent me the shirt and was
like: look how cool this shirt is. Like, he’s
the person wearing the shirt in the picture,
and he’s like, look how cool this shirt I just
made is. And so then I tweeted the shirt. And then
Irina just turned it into a merchandising scheme
to fund future scaling. So she just made a bunch
and started selling it to people. Like apparently,
like she sells like more than 10 to Anthropic
already. Just scaling lot of t-shirts, that’s
the ultimate funding model for supercomputers.
Scaling Laws, Upstream and Downstream tasks
Maybe you can like explain intuitively for
listeners that are not very familiar
to what are scaling laws in general.
Whatever your bottleneck compute data parameters,
you can predict what the performance will be
as that bottleneck is relieved. Currently,
the thing most people know how to do is
predict like the upstream performance. Like
the thing people want though is to be able
to predict the downstream performance and
upstream is what you’re like… It’s like
your literal loss function that you’re
optimizing and then downstream is just
any measure that you have of, like something you
care about, so just like a downstream dataset,
or like, I mean, usually, it’s just
mean accuracy on a downstream dataset.
And to take like concrete
examples, like for GPT-3, the
upstream task is just predict the next
word. What are the downstream tasks?
Like 190… a zillion like benchmarks that the
NLP community has come up with over the years.
Like they just evaluated like the accuracy
and like things like F1 score on all those.
And yeah, what should we care
about upstream and downstream task?
I mean, basically like up, well, we don’t
really care about upstream that much. Upstream’s
just the first thing that people knew how to
predict, I guess, like predict the scaling
of what we care about as downstream. I mean,
basically, like downstream things that improve
monotonically, they kind of can be interpreted
as like capabilities or whatever, and then
downstream stuff that doesn’t necessarily improve
monotonically often is stuff that is advertised as
alignment stuff. So like toxicity or if
you like speculate in the future, stuff
like interpretability or controllability would
be things that might not improve monotonically.
So you don’t get more interpretability
as you scale your models?
You do currently, but the class example is
like CLIP. It gets more interpretable as it
has representations that make more sense.
But you can imagine at a certain point,
it’s less interpretable because then at a certain
point, the concepts it comes up with are beyond
human comprehension. Like now it’s just how
like dogs can’t comprehend calculus or whatever.
Defining Alignment and AGI
Yeah, when you mention alignment, what’s
the easiest way for you to define it?
I mean, the Anthropic definition’s pretty
practical. Like we want models that are
helpful, honest, and harmless, and
that seems to cover all the like
weird edge cases that people can like come
up with on the Alignment Forum or whatever.
Gotcha, so it is not like a technical
definition. It’s more a theoretical one.
Yeah, yeah.
So would you consider yourself an alignment
researcher or more like a
deep learning researcher?
I’d say just a beneficial AGI researcher.
That seems to cover everything.
What’s AGI?
The definition on NASA website’s pretty
good. Highly autonomous systems that
outperform humans at most
economically valuable tasks.
AI Timelines
When do you think we’ll get AGI?
I’ll just say like, it depends
mostly on just like compute stuff,
but I’ll just say 2040 is my median.
What’s your like 10% and 90% estimate?
10%, probably like 2035.
Recent Progress: AlphaCode, Math Scaling
I think there’s been a week where we got
DALL-E 2, Chinchilla, PaLM. Did that
like update your models in any way?
The one that I thought was the like… was
the crazy day was the day that AlphaCode
and the math-proving thing happened on the same
day, because like, especially the math stuff,
like Dan Hendricks has all those slides where he
is like, oh, math has the worst scaling laws or
whatever, but then like OpenAI has like the IMO
stuff. So like at least according to like Dan
Hendricks’ slides, whatever, that would’ve been
like, something that took longer than it did.
So when you mentioned the IMO stuff, I think
it was like at problem from maybe 20 years ago,
and it was something that you can like
do with maybe like two lines of math.
I agree they weren’t like super,
super impressive, but it’s more
just the fact that math is supposed to have
like the worst scaling supposedly, but like
impressive stuff’s already happened with math now.
Why is math supposed to have the worst scaling?
It’s just an empirical thing. Like
Dan Hendricks has that like math
benchmark thing and then he tried to do
some extrapolations based on the scaling
of performance on that. But the amount
of computing data we currently have,
it’s already like doing interesting
stuff was kind of surprising for me.
I think in the paper, they mentioned that the
method would not really scale well because of,
and some infinite actions base when
trying to think of like actions.
Yeah.
So yeah, I didn’t update it. I was like,
oh yeah, scaling will be easy for math.
I didn’t update it as easy, but
just easier than I had thought.
The Chinchilla Scaling Law
Okay, related to scaling,
the paper by DeepMind about the Chinchilla
model was the most relevant, right?
Yeah, I thought it was interesting. Like,
I mean, you probably saw me tweet it,
like that person on Eleuther
Discord that was like, oh wait,
Sam Altman already said this like six months
ago, but they just didn’t put it in a paper.
Yeah, he said that on the Q&A, right?
Yeah, yeah.
Yeah, he said something like we shouldn’t,
our models will not be like much bigger.
Yeah. He said they’ll use way more
compute, which is analogous to saying,
there you’ll train a smaller
model, but on more data.
Can you like explain the kind
of insights from scaling laws
between like compute model size, and then like
what’s called like the Kaplan Scaling law?
It was originally something about computing.
If your compute budget increase a billionfold,
your model size increases a millionfold and
your dataset size increases a thousandfold.
And now it’s something like, I know it’s like
one to one, but I don’t remember like how big
the model size to like compute ratio was. I know
like the model-to-data ratio is one to one now,
but I don’t remember what the compute-to-model
ratio is, the new compute-to-model ratio is.
That’s also what I remember, and
I think like the main insight
from the first thing you said from the Kaplan
law is that like model size is all those matters
compared to dataset and
for a fixed compute budget.
Yeah, the narrative with the Kaplan one was model
size, like compute is the bottleneck for now until
you get to the intersection point of the compute
scaling and the data scaling, and at that point,
data’s gonna become more of a bottleneck.
So compute is the bottleneck now.
What about like having huge model?
But yeah, yeah. That’s like, because like they
were saying that because model size grows so fast.
So like to get the bigger models, you need more
compute rather than like, you don’t need more data
‘cause like you don’t even have enough compute
to like train a large model on that data yet,
with the current compute regime… was the
narrative of the first of the original Kaplan
paper. But it’s different now because like the
rate at which you should be getting data given,
like the rate at which your data charge should be
increasing given your compute budget is increasing
is a lot faster now, like using the Chinchilla
scaling law. For some increasing compute size,
you’re gonna increase your model by a certain
amount, and the amount that you’re dataset size
increases is like a one-to-one relation to the
amount that your model size increases. I don’t
remember what the relation between model and
compute was, but I know that now the relation
between model and dataset size is one to one,
between model size and dataset size is one to one.
And the main size is that now we can just
have more data and more compute, but not like
a lot of more compute. We just need the same
amount as more compute. So we can just like
have to scrap the internet and get more data.
It just means like to use
your compute budget optimally,
the rate at which your dataset
size grows is a lot faster.
Does that make you more confident that we’ll
get like better performance for models quicker?
Maybe for like YouTube stuff, because YouTube,
we’re not bottlenecked by data. We’re bottlenecked
by compute, whatever. But that implies the
model sizes might not grow as fast for YouTube
or whatever. But for text, we’re probably gonna be
bottlenecked by… It means we’re probably gonna be
bottlenecked like text and code by the dataset
size earlier than we thought. But for YouTube,
that might like speed up the unsupervised
video on all of YouTube, like timeline stuff.
Limits of Scaling: Data
Yeah, so I’m curious when
do you think about like how much are
we bottlenecked by data for text?
Yeah, I asked Jared Kaplan about
this, and he said like, “Wait,
okay. “It’s 300 billion tokens for GP3.” And
then he said like, library of Congress, whatever,
could be 10 trillion tokens or something like
that. And so like the most pessimistic estimate of
how much like the most capable organization could
get is the 500 billion tokens. A more optimistic
estimate is like 10 trillion tokens is how many
tokens the most capable organization could get,
like mostly English tokens.
So how many like orders of magnitude in
terms of like parameters does this give us?
I don’t remember what the… Like I haven’t
calculated it. Like I remember I kind of did it
with the old one, but I haven’t done it with the
new Chinchilla one. But I mean, you said this in
your thing today or whatever, like we probably
are gonna be bottleneck by the amount of code.
I was essentially quoting Jared Kaplan’s video.
Code Generation
Yeah, yeah, but he, I mean, he’s right.
I’m kind of wondering what’s philanthropic
thinking of Adept, because Adept’s like doing
the training all the code thing, and Adept was
gonna do all the train on all the code thing,
and they’re like, oh crap, we got another
startup doing the train on all the code stuff.
Yeah, so I think you said that if you remove
the duplicates on GitHub, you get some amount
of tokens, maybe like 50 billion tokens, 500, I’m
not sure. Maybe 50 billion. Don’t put me on that.
Yeah.
And yeah, so the tricks will be data
augmentation… you’re like applying the
real things to make your model better, but it’s
not clear how do you improve performance? So my
guess would be you do transfer learning, like
you train on like all the different languages.
That’s definitely what they plan on
doing, like you see the scaling lots
for transfer paper is literally pre-train
on English and then fine-tune on code.
My guess is also that like, if you get a bunch
of like the best programmers in the world
to use co-pilot and then you get like feedback
from what they accept, you get higher quality
data. You get just like, oh yeah, this work
just doesn’t work. And so you have like
1 million people using your thing 100 times a
day, 1,000 times a day, then that’s data for free.
I mean, I view that part kind of as like
the human feedback stuff is kind like the
alignment part is the way I view it. I mean,
then there’s some people who like say, oh,
there might be ways to get like better
pre-training scaling if you have like
humans in the loop during the pre-training,
but like, no one’s really figured that out yet.
Well, don’t you think like having
all this telemetric data from
GitHub cooperatives is you can use it, right?
Yeah, yeah, but I almost view it as like
that it’s like used for alignment, like for
RL from human preferences.
Okay. Gotcha. Yeah, I think the other thing they
did for improving GPT-3 was just having a bunch of
humans rate the answers from GPT-3 and then like
that’s the paper of instructivity. I think like
they had a bit of humans and it kind of
improved the robustness or not for business, but
alignment of the answer somehow. Like
it said less like non-ethical things.
Yeah. I mean it’s like people downvoted
the non-ethical stuff, I think.
Youtube Scaling, Contrastive Learning
Exactly, yeah. And to go back to YouTube,
why is scaling on YouTube interesting?
Because there’s unlimited data?
Yeah, one, you’re not banned, but I mean, the gist
is YouTube’s the most diverse, like simultaneously
diverse and large source of
like video data basically.
And yeah. So for people who were not used
to or thinking, what’s the task in YouTube?
Yeah, it could be various things. Like it might be
like a contrastive thing or it might be a predict
all the pixels thing. Like, I mean, so like
at least places like Facebook seem to think
like contrastive has better downstreams scaling
laws, so it’s gonna be a contrastive type thing.
What’s contrastive type thing?
Like you want representations that have similar
like semantic meaning to be close together,
like have low cosign similarity, like in
latent space. So basically, like maximize
the mutual information between views. Like
it’s kind of hard to explain without pictures.
So you’d say that your model takes a video,
like all of the videos and views as input?
Frames that were close together like in time, it
tries to maximize the mutual information between
them via maximizing cosign similarity between
the latents of like a resonant encoder or
whatever that encodes the images for both of those
frames that were next to each other, like in time.
So he tries to kind of predict
correlations between frames in some kind of
latent space from a resonance?
Yeah, yeah. In the latent space, you want frames
that were close to each other in time to have
similar, like maximize the cosign similar between
the latent space between the latent between the
hidden layer output by the like resonance that
took each of those in each of those frames in.
And at the end of the day, you want something that
is capable of predicting how many frames in lens.
Kind of for, well, the like philosophy with
like the contrastive stuff is we just want a
good representation that’s useful for downstream
tasks or whatever. So like you don’t actually
like, there’s no like output really.
It’s just you’re training a latent space
or whatever that can be fine-tuned
to downstream tasks very quickly.
What are the useful downstream
tests, like robotics?
Yeah, yeah. Like there’s a zillion
papers on like people pre-train on
do some pre-train contrastive thing in like an
Atari environment, and then they show like, oh,
now we barely need any RL steps to like fine-tune
it or whatever and it can like learn RL really
quickly after we just did all this unsupervised
contrastive, like pre-training or whatever.
And yeah, wouldn’t your model be kind of
shocked by the real world when you just
like show him like YouTube videos all the time
and then you trust the robot with like a camera?
Kind of not. I mean, ‘cause there there’s
like everything on YouTube. They got like
first person egocentric stuff, they got third
person stuff. Like it’ll just like realize which,
like whether it’s in first or third person pretty
quickly. I feel like it just infers the context.
Like now I saw GPT-3 just for the context, it’s
in, ‘cause it seemed like every context ever.
Gotcha. So I was mostly thinking
about like entropy of language.
If it’s literally like a video generative model,
then you can do like just the perfect analogies,
GPT-3 or whatever. It gets a little trickier
with like contrastive stuff, but yeah, I mean
either one. I mean the analogies
are pretty similar for either one.
So one of the things about the scaling
laws papers and the role of scaling laws,
there was some different exponents for text.
Yeah.
Scaling Exponent for Different Modalities
What do you think is the exponent for
video? Would it be like much worse?
I know the model size. The model size relation
was the big point of the scaling laws. For
autoregressive generative models, the paper says
that the rate at which the model size grows,
given your compute budget grows,
is the same for every modality.
So that was kind of like, that’s
like a big unexplained thing.
Like that was the biggest part just of that paper
and no one’s been able to explain why that is yet.
So there might be some universal law where scaling
goes for all modality and nobody knows why.
Just stuff. The rate at which
your model size grows given
your compute budget is increasing
is the same for every modality,
which is kind of weird and no one, like I
haven’t really heard a good explanation why.
Who do you think will win
the video prediction race?
AGI Race: the Best Funding
Model for Supercomputers
The person who wins AGI is whoever has the best
funding model for supercomputers. Whoever has
the best funding model for supercomputers wins.
Like, I mean yet to assume all entities are like,
they have like the nerve, like we’re gonna do
the biggest training run ever, but then given
that’s your pre-filter, then it’s just whoever
has the best funding models for supercomputers.
So who is able to spend the most money?
So would it be USA, China, Russia?
Yeah, yeah, it might be something. I mean, my
guess is like China’s already, like they already
have this joint fusion of industry government
and academia via the Beijing Academy of AI
in China. So my guess is like at some point,
like Beijing Academy of AI and be like, look,
we just trained like a 10 to the 15 parameter
model on all of YouTube and spent like $40 billion
doing it. And then at that point, Jared Kaplan’s
gonna be in the White House press conference room,
be like, look, see these straight lines on
log log pots, we gotta do this in the USA now.
Right, right. But how do you
even spend that much money?
By making people think if
they don’t, they’ll no longer be the
superpower of the world or whatever. Like
China will take over the world or whatever. Like
it’s only like a fear. It’s only a fear thing.
From looking at the PaLM paper from Google, they
seem pretty clever on how they use their compute.
You mean the thing where they have like the
two supercomputers that they
split it across or whatever?
Right. I think TPU pods or
something, they call it.
Yeah, yeah.
So it didn’t seem like they
spent more money than OpenAI.
So they tried to be more careful somehow. So my
model of like people spending a lot of money is.
Like most entities won’t be willing to like do the
largest training when they
can, given their funding.
So maybe China, but I see Google as being more
helpful because of they do it
on paper, but maybe I’m wrong.
Jared Kaplan says like most like Anthropic and
OpenAI are kind of unique in that they’re like,
okay. We’re gonna like throw all our funding
into this one big training run. But like Google
and like ‘cause Google and Amazon, they have like
he said like at least, 10X or like 100X times the
compute that OpenAI and Anthropic have, but they
never like use all the compute for single training
runs. They just have all these different teams
that use to compute for these different things.
Yeah, so they have like a different
hypothesis. OpenAI is like scale is
all that matters, somehow that
they’re secrets itself and-
Yeah, it’s something like that.
You just let scale things and we
are going to get better results,
and Google is maybe there’s more bureaucracy
and it’s maybe harder to get a massive budget.
Private Research at Google and OpenAI
Yeah, it’s weird though, ‘cause
Jeff Dean’s latest blog
post, it summarizes all the
Google’s research progress mentions like scaling
and scaling while it’s a zillion times. So that
almost implies that like they’re on the scales.
All you need bandwagon too. So I don’t know.
They probably know, but then the question
is how like private things are and
maybe there’s stuff we don’t really know.
I know a bunch of Google said like,
yeah, we have language models that
are way bigger than GPT-3, but
we just don’t put ‘em in papers.
So you’ve talked to them like privately
or is it just, they said online?
I just I’ve heard things from people and
that’s feasible. I’m not just disclosing
what I got that information from, but
that’s just what I’ve heard from people.
So as we’re on like gossip, I think like
something that was around on the internet,
like right when GPT-3 was launched was that Google
was like reproduced it in a few months afterwards,
but they didn’t really talk about it publicly. I’m
not sure about what to do with this information.
I know like the DeepMind language models
papers that they were a year old when
they finally put ‘em out on archive or
whatever, like Gopher and Chinchilla.
They had the language model finished
training a year before the paper came out.
So we should just like assume all those big
companies are just like throwing papers when
they’re like not relevant anymore when
they have like the other paper already?
Maybe, but yeah. I don’t know why
it was delayed that much. Yeah,
I don’t know what the story is.
Why it was delayed that long.
People want to like keep their advantage, right?
I guess, but I mean like I feel like GPT-3,
they threw the paper on arXiv pretty
soon after they finished training GPT-3.
How do you know?
Yeah, I don’t, but I mean,
yeah, I don’t. But ice, it didn’t. Yeah,
maybe there was a big delay. I don’t know.
So I think you could just like retrace all
Sam Altman tweet and then like you read the
next paper like six months after and you’re
like, oh yeah, he tweeted about that. Like
sometimes the tweets like, oh, AI is going to be
wild, or oh, neural networks are really capable
of understanding. I think you tweeted that like
six months ago, like when they discovered GPT-4.
OpenAI is like when Ilya tweeted the consciousness
tweet, they’re like, goddamn, GPT-4 must be crazy.
Yeah, neural networks are in
some ways slightly conscious.
Yeah, yeah, that was the funniest quote.
Yeah, I think people at OpenAI know things we
don’t know yet. They’re all like super hyped.
And I think you mentioned as
well that at least privately that
Microsoft has some deal with OpenAI and so they
need to some amount of money before 2024, like.
Oh yeah, yeah, yeah, yeah. I mean, right,
right. When the Microsoft deal happened, like
Greg Brockman said, “Our plan is to train
“like a 100 trillion parameter model by 2024.”
Okay, so that’s in two years?
I mean, that was in 2019,
but maybe they’ve changed their mind after like
the Chinchilla scaling lot stuff, I don’t know.
Why Ethan did not update that much from PaLM
Right. And so you were not like impressed
by PaLM being able to predict to like do
logic on airplane things and explain jokes?
In my mind, like the video scaling was like
a lot worse than text basically. That’s the
main reason why I like AGI will probably take
longer in the five years or whatever in my mind.
Okay, so we need, so if we just
have text, it’s not enough to
have AGI. So if we’re a like a perfect Oracle
that can like talk like us, but it’s not able to
do robotic things, then we don’t have AGI.
Yeah.
Well, I guess my main like is mostly
like coding. So if we get like coding,
like Codex or comparative, that gets really good,
then everything accelerates and engineers
become very productive, and then like.
I guess if you could say like, engineers get
really productive at making improvements in
hardware, then like, maybe that would, like,
I get how that would be like, okay. Then it’s
really fast. Like in my mind, at least at
the current, I don’t see the hardware getting
fast enough to be far enough on the YouTube
scaling lot in less than five years from now.
Thinking about hardware, we’re just
like humans, Googling things and using.
Yeah, yeah. I get what you’re saying.
Like you get the Codex thing and then
we use Codex or whatever
to design hardware faster.
You mentioned you have like DALL-E,
but like for designing chips.
I mean, Nvidia already uses
AI for designing their chips.
That doesn’t make you think of
timelines of 10 years or closer.
It may be 10 years, but not five years.
The thing I’m trying to figure out is like,
try to get like a student researcher gig
at like someplace so that I can just get
access to the big compute during the PhD.
Oh, so that’s your plan. Just get out of compute.
Yeah, I mean, as long as I have big compute,
it doesn’t matter where I’m a PhD. I mean,
it kind of matters if you’re like trying
to start an AGI startup or whatever, but
safe, safe, safe AGI startup.
We’re kind of on record, but
I’m not sure if I’m going to
cut this part. So you can say unsafe, it’s fine.
Yeah, no, no, no. I mean, I don’t even
phrase. I just phrase it as beneficial AGI.
You were spotted saying you wanted
unsafe AGI the fastest possible.
Thinking about the Fastest Path
No, no, no. The way I phrase it is
I think I explained this last time, you have to
be thinking in terms of the fastest path, because
there is like extremely huge economic and military
incentives that are selecting for the fastest
path, whether you want it to be that way or
not. So like, you gotta be thinking in terms of,
what is the fastest path and then how do you like
minimize the alignment tax on that fastest path?
‘Cause like the fastest path is the way
it’s probably gonna happen no matter what,
like, so it’s about minimizing the
alignment techs on that fastest path.
Or you can just throw nukes everywhere
and try to make things slower?
Yeah, I guess, but I mean the people who are
on the fastest path will be like more powerful,
such that like, I don’t know, such
that they’ll deter all the nukes.
So you want to be, okay, so you
want to just like join the winners.
Like if you join the skiing team at Google.
Thing I’ve been trying to brainstorm
about is who’s gonna have the fastest,
who’s gonna have the best funding models for
supercomputers, ‘cause that’s the place to go
and you gotta try to minimize
the alignment tax at that place.
Makes sense. So everyone should infiltrate Google.
Yeah, so whatever place ends up with
the best funding model of supercomputers
try to get as many weird alignment people
to like infiltrate that place as possible.
So I’m kind of happy having a
bunch of EA people at OpenAI now,
because they’re kind of
minimizing the text there, but…
Yeah, I kind of viewed it as all the EA
people left, like ‘cause Anthropic was
like the most extremist EA people at OpenAI.
So I almost viewed when Anthropic happened
a bunch of EA people. I view as that like EA
almost leaving OpenAI when Anthropic happened.
Some other people came, right?
Like who?
I don’t know. Richard Ngo.
Oh, okay, okay. Yeah, yeah.
It’s like a team on like predicting the future.
Yeah, yeah. I wanna know what
the Futures Team does ‘cause
that’s like the most out there team. I’m
really curious to what they actually do.
Maybe they use their GPT-5
model and predict things.
Right, ‘cause I mean like DALL-E, like you
know about the Foresight Team at OpenAI, right?
They were trying to predict
things as well, like forecasting.
Yeah, that’s where all this scaling lot stuff came
from was on the Foresight Team at OpenAI. They’re
gone now because they became philanthropic. But
like a team called like the Futures Team that
almost has a similar vibe to like a team called
the Foresight Team. So I’m kind of curious.
But then there’s just like doing more governance
things and optimal governance and maybe economics.
That’s what it’s about, governance and economics.
The guy like Richard Ngo
is doing governance there.
Okay.
Predicting how the future works,
I think is in his Twitter bio.
Yeah, yeah, but I mean, that’s
somewhat tangential to governance,
like that almost sounds like something Mike Rick
Kurtz, I would say, I’m predicting how the future.
My model is like Sam Altman,
as like they have GPT-4.
Like they published GPT-3 in
2020. So it’s been like two years.
Yeah.
And they’ve been talking about like
in their Q & A about like treacherous
results or something like one year ago. So now
they must have access to something very crazy
and they’re just like trying to think like how
do we operate with like DALL-E 2 and their GPT-4
they have in private and how they do something
without like for him in the world? I don’t know.
Maybe they’re just like trying to predict like
how to make the most money with their API or.
You’re saying like if they release it,
it’s like an infohazard? ‘Cause in my mind,
GPT-4 still isn’t like capable enough
to F up the world, but you could argue,
it’s like capable enough to like
be an infohazard or something.
Imagine you have access to
something that has the same
kind of gap between GPT-2 and GPT-3, but like for
GPT-4 on like understanding and being general.
And you don’t want everyone
else to copy your work.
So you’re just going to keep
it for yourself for sometime.
A Zillion Language Model Startups from ex-Googlers
Yeah, but I feel like that strategy is already
kind of screwed. Like you know about how like
a zillion large language model, like a zillion
Googlers have left Google to start large language
model startups. Like there’s literally three
large language model startups by ex-Googlers now.
OpenAI is like a small actor in this now because
there’s like multiple large language model
startups founded by ex-Googlers that all like
that all were founded in the last like six months.
Like there’s a zillion VCs throwing money
at large language model startups right now.
The funniest thing, like Leo Gao, he’s like, we
need more large language model startups because
the more startups we have, then it splits up all
the funding so no organization can have all the
funding to get the really big supercomputer.
So we just need thousands of AI during its
final startups. So no one can hoard all the
funding to get the really big language model.
That’s the, yeah, with the AI model,
you just like do open source. So like
there’s like more startups. And so all
the funding gets splitted, I guess.
Yeah, you could view OpenAI was like extra big
brain. We need to do. We need to like release
the idea of our joiners models onto the world such
that no organization could have enough compute to
be such that all the compute gets more split up,
‘cause a zillion, our joiners model
startups will show up all at once.
That’s yeah, that’s the best idea ever. So do you
have like other gossips besides like Google’s?
Did you post something on Twitter
about people leaving Google?
Yeah, I posted a bunch of stuff. Well, I mean, and
also like you saw the… I mean it’s three startups,
adept.ai, character.ai, and inflection.ai.
They’re all large language model startups
founded by ex-Googlers that got a zillion dollars
in VC funding to scale large language models.
What’s a zillion dollars like?
Like greater than 60 million. Each
of them got greater than 60 million.
So did they know about something we
don’t know? And they’re just like
get money to replicate what Google does?
Well, I mean, most of ‘em,
they were famous people like
founder of DeepMind scaling team. Another
one is the inventor of The Transformer.
Another one was founded by a different
person on The Transformer paper. Like,
so I mean, in some ways, they have more
clout than like OpenAI had or whatever.
But they don’t have like the
engineering and old infrastructure.
No, they kind of do. Like, a lot of ‘em,
they were like the head of engineering for
scaling teams at like DeepMind or Google.
So there’s like another game that is in
private at Google and they’ve been scaling
huge models for two years. and they’re just like,
Yeah, something like that.
Starting startups with their knowledge and they’re
just scaling and we;re just like, like peasants
like us talk about papers that are released
one year after and then when you turn them out.
Yeah, yeah. I guess that’s, I mean, I don’t know
how long these delays are. I mean, in my mind,
like, yeah. I guess you could view it as a delay
thing, ‘cause like in my mind it’s just like,
yeah, you’re right, you’re right.
It’s probably delayed by a year, yeah.
So yeah, that makes me less confident about-
Oh shit. You look like a clone
of Lex Fridman from the side.
What?
When your face is like sideways, you
look like a clone of Lex Fridman.
Yeah.
Like, ‘cause your haircut’s
like identical to his when
I’ll take that as a compliment… I started working
out. So yeah, Ethan Caballero, what’s the meaning
of life?
Probably just maximize the flourishing of all
sentient beings, like a very generic answer.
Right. So I’ve done my Lex Fridman
question. Now I’m just basically him.
Yeah.
Ethan’s Scaling Journey
Maybe we can just go back to
like stuff we know more about like your work and
because you’ve been doing some work on scaling.
Yeah.
So like more general, like why are you
kind of interested in scaling and like how
did you started on doing research on that?
I mean, I knew about the body paper when
it came out. Like I remember I was at this
like Ian Goodfellow talking in 2017 and he was
hyped about the body paper when it came out.
Which paper?
The deep burning scales, predictably, empirically,
yeah, it came out 2017 and then I kind,
I just, that was just on the back burner and I
kind of just stopped paying attention to it after
a while. And then like Aran Komatsuzaki was
like, no, dude, this is the thing. Like this
is gonna take over everything, and this was
like in 2019 when he was saying that. And then,
yeah. So then when the scaling laws papers got
like re-popularized through like the OpenAI stuff,
then I kind of like caught onto it a little
bit early via like talking with Aran.
I think in 2019 was also
when GPT-2 was introduced.
But that was kind of before, like
that was before like the scaling
law stuff kind of got popularized.
Right, scaling laws paper is 2020.
Yeah, the very end of 2020.
All right. No, no, no. Oh no,
no. The scaling law paper was the very end
of 2000. It was the very beginning of 2020.
And you were already on this
killing train since 2017.
I was aware of it, but I didn’t, like, I
was kind of just neutral about it until like
2000, like probably the middle of 2019.
Making progress on an Academic
budget, Scaling Laws Research
And yeah, now you are kind of
interested in scaling because
it’s useful to predict kind of what
the whole field of AI is going.
And also it just, it’s I think people
underestimate how easy it is to be contrived
if you’re not paying attention to scaling trends
and trying to like extrapolate the compute budgets
and data budgets that are like, well, the compute
data and data budgets like five years from now.
Yeah, if you’re a huge company that does a lot of
budget, but maybe if you’re just a random company,
you don’t really care about scaling law that much.
Yeah, yeah. Or if you’re like in
academia currently or whatever,
like a zillion papers that like fancy conferences
are like, here’s our inducted bias that helps on
like our punny academic budget. And we didn’t
test any of the scaling asso tos to see if it’s
like useful when you’re training a trillion
parameter model on all of YouTube or whatever.
You’re on an academic budget as far as I know. So
how do you manage to do experiments in scaling?
There’s like the scaling on narrative. That’s
like, oh, you don’t need the big budget
to do because you can just predict what the
outcomes will be for the large scale experiments,
but that’s at least current. At least
when that narrative got popularized,
it was mostly for upstream like scaling. But the
thing everyone cares about is downstream scaling.
AI Alignment as an Inverse Scaling Problem
Yeah, so if we go back for a minute on
like your work in alignment, how
do you think your work on scaling
or generalization like kind of
fits with the alignment problem?
Basically, all alignment, I guess this
triggers the hell outta some people. But
all alignment is inverse scaling problems.
It’s all downstream inverse scaling problems.
So it’s just in my mind, all of alignment is
stuff that doesn’t improve monotonically
as compute data and parameters increase.
There’s a difference between not improving and
inverse scaling. Inverse
scaling goes badly, right?
Yeah, yeah, yeah. But I just said not improved
monotonically, ‘cause like sometimes there’s
certain things where like it improves for a while,
but then at a certain point, it gets worse. So
like interpretability and controllability are the
two like kind of thought experiment things where
you could imagine they get more interpretable
and more controllable for a long time until they
get super intelligent. At that point. they’re
like less interpretable and less controllable.
Do we have benchmarks for controllability or?
Like just like just benchmarks that rely on
prompting is a form of like a
benchmark of controllability.
And kinda to summarize your take, if we
were able to just scale everything well
and not have this inverse scaling
problem, we would get like
interpretability and controllability
and everything else by just like good
scaling of our models. And so we’d get like
alignment kind of by defaults for free?
Yeah. I mean, I guess, I mean like there’s
stuff besides interpretability, controllability,
like those are just the examples. Like what you
said, you asked like what’s an example where
like the reason I said, I phrased it as
alignment is when I said inverse scaling,
I said things that don’t improve monotonically,
‘cause I just wanted to say like,
yes, there’s obvious examples where it gets
worse the entire time, but there’s some you
could imagine where it gets good for a long time,
and then a certain point, then it starts getting
drastically worse. I said, all of alignment
can be viewed as a downstream scaling problem.
The hard part is like Dan Hendricks and like Jacob
Steinhardt say like, then the hard problem though
is like measurement and like finding out what
are the downstream evaluations ‘cause say you
got like some like fancy like deceptive AI that
wants to like a treacherous turn or whatever.
Like how do you even find the downstream
evaluations to know whether it’s gonna like
try to deceive you or whatever? ‘Cause like when
I say, it’s all a downstream scaling problem,
that assumes like you have the downstream test,
the downstream like thing that you’re evaluating
it on. But like if it’s like some weird deceptive
thing, that’s like, it’s hard to even find what’s
the downstream thing to evaluate it on to like
know whether it’s trying deceive or whatever.
So there’s no like test lost on
this deception. We don’t know
for sure how to measure and have
a clear benchmark from this.
Yeah, it’s tricky. I mean, and some
people say like, well, that’s why
you need better interpretability. You need to
like find the deception circuits or whatever.
Knowing that we don’t know yet, like all the
different benchmarks and metrics for misalignment,
don’t you think that your
work on scaling can be bad
because you’re actually
like speeding up timelines?
Predicting scaling laws,
Useful AI Alignment research
Yeah, I get the like infohazard
point of view, but like
in my mind, like whether you wanna do all
capabilities or alignment stuff that stands
the test of time, you need really
good downstream scaling prediction.
Like, say you came up with some like alignment
method or whatever that mitigates inverse scaling,
like you need the actual functional form to know
whether that thing will like keep mitigating
inverse scaling when you get to like a trillion
parameters or whatever. You get what I mean?
I get you but like on a differential
progress mindset, like Jared Kaplan or
someone else will come up with those
functional forms without your work.
I don’t know, I don’t know. I
mean, that’s the thing though, like
Anthropics (ERRATUM: it’s actually a gift, and the
the merch was not sent at the time of the podcast)
got that paper like predictability and surprise
and generative models and they’re just like,
it’s unpredictable. We can’t predict it. And
I’m like, ah, you guys, nah, I don’t believe.
Right, so you’re kind of publishing
papers when you’re in advance
because those companies are
not publishing their results?
I don’t know. I don’t. Yeah, I don’t even, I don’t
know if Anthropic does the delay type stuff that
OpenAI supposedly does, but
maybe they do, I don’t know.
And you were just like drawing
infohazard by publishing those laws?
I mean, in my mind, whether or not, I get the
argument, oh it, if you wanna do capabilities
work that stands a test of time or alignment work
that stands a test of time, in my mind, everything
that people are doing in alignment will be very
contrived without the functional form too though.
So it’s like alignment can’t make progress without
it either. So it’s like, you get what I mean?
Another kind of view on that is that
if people do impressive deploying or
ML board and they’re also interested
in alignment, it’s still a good thing.
Like let’s take even through AI. Even if they
open source their model because they did something
impressive and they talk openly about alignment
under Discord and gets like a lot of people
that are very smart, interested in alignment. So
if you publish something and you become like a
famous researcher, something in two years and you
talk about alignment in two years, then it’s fine.
I sort of tweet stuff about alignment,
I think. Yeah, I mean, I retweet
stuff about alignment at least.
Ajeya Cotra’s report, Compute Trends
So if we go back to thinking about predicting
future timelines and kind of scaling, I’ve read
somewhere that you think that in the next few
years, we might get billion or trillion times of
more compute, like 12 orders of magnitude more.
Yeah, I mean, so the Ajeya Cotra report said like,
it’s gonna max out probably at 10 to the 12 times
as much compute as like the amount of compute in
2020, probably like 2070 or something like that.
The one issue I have with the JS model is
that like, she does, what does she do? It’s
like it’s flops per dollar times willingness
to spend its total flops that are allocated to
pre-training runs. Problem is like, for the
big like foundation models, like 10 of the
15 perimeter miles of the future or whatever,
you’re probably gonna need high pie like memory
bandwidth between all like memory bandwidth and
compute bandwidth between all the compute, which
means it has to be on a supercomputer. So it’s not
just the flaps. It basically what really matters,
at least if you’re assuming it’s like big, like 10
of the 15 parameter foundation models or whatever,
like the speed of the fastest supercomputer is
what matters, not just the total flaps that you
can allocate, because if like all the flaps
don’t have good communication between them,
then they aren’t really useful for training like
10 of the 15 parameter model or whatever. Once you
get to 10 of the 15 parameters, like there isn’t
much reason to go beyond that. And at that point,
then you just have multiple models with
10 of the 15 parameters and they’re like
doing some crazy open ended, like Ken Stanley
stuff and a multi-agent simulator after you do
that. Like if they mentioned became like you do
the 10 of the 15 parameter model feature and all
YouTube, and then after that, you’ll have like
hundreds of 10 of the 15 parameter models that
all just duke it out in like a Ken Stanley
open-ended simulator to like, get the rest of
the capabilities or whatever, like once they’re
in the Ken Stanley open-ended stimulator,
then you don’t need high compute bandwidth
between all those individual, like 10 of the
15 parameter models, like duking it out in
the simulator. They can just, each one, they
only needs like 10. It only needs high compute
bandwidth between like its own parameters. Like,
it doesn’t need high compute bandwidth between
itself and the other like agents or whatever.
And so in there, the flops where you could use
all the flops for like the multi-agent simulation,
but you only need high compute
bandwidth within each agent.
So you need a lot of bandwidths to train
models because of the prioritization thing,
but you only need flops to simulate
on different things at the same time?
Yeah, you only need high compute bandwidth
within an individual brain, but like if you
have multiple brains, then you don’t need
high compute bandwidth between the brains.
And what was that kind of simulator
you were talking about, the Kenley?
Like Ken Stanley, the open-ended guy.
I haven’t seen that.
Ken is like the myth day objective
open-endedness, like Can Stanley’s, Jeff Cones,
like all that stuff. It’s like, I don’t know. Just
Google, like Can Stanley open ended at some point.
You’ve probably heard of it, but it’s not
like registering what I’m referencing.
Optimism, conclusion on alignment
Okay, so maybe one kind of last open-ended
question. On a scale from Paul Christiano, Eliezer
Yudkowsky, Sam Altman, how optimistic are you?
Definitely not like Eliezer, or a doomer type
person. I guess probably Paul Christiano is most
similar. I mean, I feel like Paul Christiano
is in the middle of the people you just said.
Right. Yeah. So you are less
optimistic than Sam Altman?
Well, yeah, I mean, basically, I think
deceptive AI is probably gonna be really hard.
So do you have like one less
monologue or sentence to say about
why scaling is a solution
for all alignment problems?
Like just all alignment can be viewed as an
inverse scaling problem. Like it all revolves on
just mitigating inverse scaling, but also you have
to make sure you have like the right downstream
things that you’re evaluating, like the inverse
scaling and like part of what makes it hard is
like you might need to do like fancy thought
experiments on alignment, like counterintuitive
thought experiments on alignment forum to find
what are the downstream… to find what are the
the downstream tests that
you should be evaluating.
Like whether or not there’s
inverse scaling behavior on those.
Awesome, so we get the good
version, as last sentence,
and that’s our conclusion. Thanks
Ethan for being on the show.