219. Misconceptions on discontinuous takeoff

hello and welcome to session 219
in the aisafety.com reading group
tonight we'll be discussing the article
misconceptions about continuous takeoff
by matthew barnett
matthew bunnett studies computer science
at the university of
uh california berkeley and this is a
post on alignment forum
and restaurant which is around two and a
half years old
so before we dive into the article i
would like to give a bit of
my personal view on continuity
and of course this is a mathematical
concept that was formalized by viastras
calvastras
and the way mathematicians look at
whether
a function is continuous it is not
several definitions but one of them
is that um for all points in the domain
there must be a um if someone
challenges you with an epsilon and says
is this continuous within epsilon
then you must be able to provide a delta
so that within
all points that are only delta away from
this
[Music]
point where you are wondering if the
function is continuous
uh then the value of the function must
at most be
epsilon from what it is in the middle so
let's say here
is an example where we have a function
here
and the the challenger asks us is this
continuous in the point
x equal to for an epsilon equal one half
and in this case we need to come up with
a
um so if epsilon is one half then we can
come up with a delta that is also
one half and within this area here then
the function is wholly contained within
this blue area here
so that's a rigorous definition of um
continuity of a continuous function
so in our case of course what we're
really talking about is
the trajectory of uh ai capability
so that means that the x-axis is
usually taken to be time and the y-axis
is capability in some sort
and i there are many ways to define
capability
normally when i need to define it i use
um
either the best of boston's six
cognitive superpowers
or just intelligence amplification the
ability to
recursively self-improve i believe this
is probably really
tightly correlated with general
intelligence
so it probably doesn't matter so much
but for for many practical purposes
this is far more interesting
also we'll notice that when you announce
a project
that's only a single point in time so if
you want to have
something that is defined for all points
you can say
that um then it's the uh the capability
of the most
capable system at this point in time and
in this case of course it is uh is
continues precisely in the cases
where a new project is introduced then
if the
the state of the art is here and then
there's a new project
uh with with this capability then there
is a continued
discontinuity between these two points
um
so a better um
definition of continuity as regards to
ai capability growth
would be something where we have the uh
an epsilon continuity which is a a lower
bound
and upper bound on how much the uh
the different systems are improving on
each other
so that would mean that it would be
possible for someone to make the
following statement
i am 80 sure that we will never see a 25
discontinuity meaning that we will never
see
one that is oh have a
125 percent the performance of the
previous system
and if people made this kind of
statement we could investigate
rigorously has there ever been a 25
discontinuity in the past
how large have discontinuities we've
seen and
how bad is it to be 25 more powerful
than the previous what can you do with
that in practice
and if you're only 80
sure then uh the 20 where you see more
than 25
is that like 30 percent or is that like
3 000
i think this is all these are the kind
of questions that come up naturally
if you use this um excellent definition
here that i suggest
so to go through the actual article by
matthew barnett
um so for the question of if i will
experience a discontinuity
he says the terminology is
confusing and diverse and
he provides his own definition saying
that
it will follow a trajectory that is
roughly in line with the expectations
on the negation that there is not a
point
where um
there is not a point where a single
project leaves
very much ahead so here there is much
more competent than other projects
so the standard way that normally we
graph
ai capability improvements is
this uh figure from nick bostrom's book
super intelligence chapter 4 where we
have the kinetics of an intelligence
explosion
where we're talking about the time from
we get
agi until we have something like a super
intelligence
and whether this function from here to
here will be continuous all
discontinuous
it should be noted that nick bostrom in
the book super intelligence
does talk about discontinuity a bit in a
footnote when he's talking about this
but bostrom focus very much on the
duration
from this time to this time um
so there are a number of differences
between how
matthew barnett uses and how i or
perhaps nick bostrom
licenses first whether the data points
are
projects or models like you could
imagine that
um if you have something like uh alpha
alpha go that turns into alpha star and
mu alpha zero
and mu zero do they count as
one project or do they count as several
projects
and that has a very large um
influence on how discontinuous it is
because surely if you count them all
together
as as one then it's a huge discontinuity
in the ability to play go
and it seems here that the the word
matthew barnes is using
for how much more competent it is like
much more it's quite unclear what that
is so
i would have preferred an epsilon even
like 60 or something like that
um also matthew barnett is not really
talking about capability
as much as competence plus power which
is a bit strange i would imagine that if
you have two ai systems
that are the same level of intelligence
but one of them
is controlling the stock market and the
other one is
not or one of them is controlling
nuclear weapons or something that's not
really what we're talking about
when we're talking about here even
though there are obvious ai safety
components to giving nuclear weapons to
the
to npi um also one thing from
matthew's uh uh article that isn't quite
clear
is that i believe it sounds like he's uh
disregarding this precise point the
point where the first
agi the first project that achieves
general intelligence at a
human baseline level and that's of
course a discontinuity
um well every new project is a
discontinuity
but that's one of the ones that we are
most interested in
in particular if it will do a fast
takeoff
so for the actual misconception the
first continuous doesn't necessarily
mean slow
um and matthew barnett prefers
to talk about continuous uh rather than
slower fast because he believes this is
the thing that is more strategically
relevant
to me i i think i must have a very
different conception of
uh strategy right um the the wall
uh the time uh on the clock
seems to matter very much for what uh
preparations you can do what actions you
can take
uh i mean surely in in the uh reject
your case where we have
uh like it's a takeoff that is like five
minutes
then whether it's continuous or
discontinuous doesn't matter very much
um so um matthew talks about
that the uh the capabilities that are
gained
are the ones that are really interesting
somewhat in a maybe a bit more binary
way that i would
describe it and the example that he uses
is a generative adversarial networks the
way that
given a um facial recognition uh neural
network
you can actually ask it how does a face
look like
and then you get in 2014 you got
something like this
and 2018 you get something that looks
very much
like a a photo
of a person um and this here
from over the course of four years
that's a fast
development relative to humans bostrom
would call this
a medium takeoff in the uh
ability to create uh images in this way
and um matthew barnett makes the
following
uh statement it would be unusual if
someone right now
in late 2019 produces
videos in high definition using
generative
adversarial networks so this is for
still images
for videos he believes that we are
currently have like videos
like this and it would be strange if
someone had
far better it is continuously far better
technology than what was available um
than what he believes is available so i
actually
uh looked this up to see when
was the first high definition deep fakes
created
and i managed to find a um
uh a report referenced in ieee
saying that um there were
highly realistic seamless videos so not
necessarily hd
but in fact before this
uh prediction was made um
so it references unfortunately uh uh at
that link so i can't
say precisely but if you count this that
he would be surprised if someone
did this and then in fact this had
already happened
so that counts as a i don't know if you
can say a prediction
if you say something that is already
false
um but that's um
um large power differentials can still
happen
in a continuous takeoff that's another
of these misconceptions that he is
trying to clear up and that's of course
true in the opposite
way that you can have large power
differentials even without ai
at all so of course whether ai is
continuous or discontinuous
you can still have uh differentials and
power and
the example he uses is in the ability to
produce rivals
i'm not really very fond of that example
i think
in most of the wars during the
industrial revolution
the the way that the industrialized
nations took powers
didn't have very much to do with rivals
they had more to do with a
the person's nebulous concept industry
writ large
i think so so the punch dance but uh i
don't like the specific way it's phrased
and um in particular we can see that a
single nation
or corporation can pull ahead that is
something that can happen
and has happened um and here when you're
talking about one nation or one
corporation
then it's the obvious question here
becomes
if you have two projects that are
um created by the same nation or the
same
corporation or um you know the same
small team or whatever
when are they distant enough that there
is a strategic effect
from the fact that there is an
intermediate right you can imagine
that um if for instance with alpha
go alpha zero if if all these
projects uh belong to the same
stakeholder
then there might not be so much of a
strategic effect on this
you could argue for instance that once
it's made public
then a number of people will be able to
react so
whether it's public is important whether
the safety implications can be learned
is really important whether the
stakeholders are the same
maybe uh if there are several ais that
are smart enough to cooperate
which they probably will be after lgi
then
whether there's something preventing
that or if they're capable seems like
it's really important
then there's a somewhat odd
misconception
whether a takeoff requires
that you have immolations of humans um
that was perhaps something that you
could
infer from the ai from debate with
robin hansen and he back then people who
disagreed with
uh fast takeoff probably agreed with
hansen and
that might be a reason to conflate these
two things i'm actually not entirely
sure i believe
this is what robin hansen states
now uh trying to summarize roman
henson's
ideas and predictions is
a dangerous business but i'll try in the
age of m
he sees er emulations that are
um that once you have a human then it's
really
easy to get something like an order of
magnitude improvement by just
pruning away things but that leaves a
call that is
very hard to substantially improve and
so there is something
that is roughly 10 times as fast and as
smart as a human
but not more than that and it doesn't
get smarter in any really continuous way
so instead his discussing
how would a society based on these ends
actually look um i think uh the fact
that it
uh goes up and then it kind of stops
that sounds to me like it discontinues
like
also because after the hfm it goes up
again
um so this is both slow and
discontinuous
i think but i'm not quite sure robin
hansen would agree with
how i frame his um uh his use
then there's a question about recursive
self-improvement whether that is con
compatible with continuous uh takeoff
matthew barnett believes it is
i guess probably also it is but
here we have something very on the
inside view
a very clear example of something that
could cause a discontinuity
because recursive self-improvement could
do that
and the way matthew barnett frames it is
it's sometimes argued that since growth
is exponential
a slight advantage right now might
compound to become a large advantage
over a long enough period of time and
that's of course the true general
argument um because usually this doesn't
happen i could speculate that it's
because of
things like regression to the mean um
but
my true objections to this is more with
us over a long enough period of time
because
that's the crux of the issue whether we
will have a fast takeoff or not
and in general this talk about
exponentials
is almost always wrong in the in the
context of
fast takeoff what we're talking about
here are hyperbolic functions
not exponential functions and in in this
case
this is not a kind of argument that
would be used for people
talking about fast takeoff
i guess this might be an a side but i
think
i think actually that people who are
worried about
fast takeoff usually say they're worried
about fast takeoff
and people who talk about continues or
discontinues
are not particularly worried so there
might be a
dichotomy there between people who are
worried
who will talk about uh wall clock
time and people who are not worried who
talk about whether it's continues or
discontinues
so it's actually possible that that's
the way people
talk past each other i think i only just
realized that
and that might also not be true okay uh
the last one continuous takeoff is
relevant to ai
to alignment and in this case if we get
uh as matthew says if we have rabbit
capability
again then something like treacherous
turn is a great worry that's something
that definitely could happen
and with here uh that he well he he uses
the word
rabbit so that's um
i think he means if we had discontinuous
capability gain uh because that's
kind of what what he's talking um
but he argues that if we have a
continuous takeoff
then the individual systems they won't
be able to
obtain the strategic advantage because
they are not sufficiently
powerful compared to the presses
i think uh this ignoring the possibility
that the
collaborate another thing that'll be
possible
through if we have a continuous scenario
is we can use the previous ais to match
the newer ones
and matthew barnett is very optimistic
he believes this probably means that
continuous takeoff is
incompatible with futures turns i can
see several ways when this would not be
true for instance there is the
difference between
potential capability and realized
capability it's certainly possible that
there are
uh that there is one ai who believes it
has a
five percent chance of taking over the
world and so it doesn't try
and then the next one believes it has um
like 50 and then suddenly that's one who
believes that
the probability that it can take over
the world is high enough that it
actually tries
even though the difference between them
are very low
there's also the fact that whether it's
actually possible to monitor
uh ai's whether it's possible to monitor
agents that are smart to yourself than
yourself
is potentially really hard certainly
unsolved
and there would be many reasons not to
do that for instance
if the projects are made by different
stakeholders
that would be a strong reason not to and
finally that's
that's before uh the possibility that
they could collaborate
so in the end i'm somewhat less
optimistic than matthew barnett but i do
appreciate the attempt at clarification
that is all for today thank you and see
you


AiTech Agora: Chao Zhang - Personal Autonomy in Human-AI Interaction

and so i came back to ty open my
hdi group to do my phd so
the topic was about modeling and
changing people's lifestyle behaviors
such as like encouraging people to do
more physical exercise or changing
people's daily diet
the focus of the project was on the
psychological process
i have information and the self-control
so i try to model those processes using
computational models and also sort of
implement computational models in
intelligent systems
for instance for
more accurate behavior prediction
so that's a bit different from what i
want to talk about today
and you can ask like whether my patreon
system was related to ai or not i think
it's somewhat related to ai uh so the
project was
within the data science collaboration
between gue and phoenix research so back
then in 2015 i think no one was talking
about ai yet it was a big data data
science was the password back then
so i also worked some people from
computer science so i used computational
modeling and machine learning in my own
work so that's good connection with ai
in terms of topical autonomy really not
so much through my phd i included maybe
a little bit discussion on what are the
ethical implications of this behavior
change system and people's autonomy is
one of them but that was a very brief
discussion
and after my phd i decided to go to go
to a different environment so i went to
utrecht to do a postdoc in the
department of psychology i always wanted
you to be in the mukha or traditional
psychological department for a while so
that was the motivation
i will talk a little bit more about the
human ai
project
in a bit
and then last october i
went back to my old research group
as assistant professor
and so
right now my i have a couple of
different research nights so like i
still continue to do a bit
work on habit and soft control and try
to model those processes and to see how
we can use those models for for behavior
change
there are some topics more relating to
ai
such as decision making and autonomy
issues you human ai interaction
also started to work with some other
people in my group are human robot
injection especially focusing on
emotions
and i have also the idea maybe to use
social robot also for behavior change
purposes then that's why new area wants
to sort of us invest some time
uh in consumption may be nice to mention
that i'm also developing a new human ai
interaction course i don't know if
there's anything similar here in
tutorial so we'll be happy to discuss
with you also about
similar
projects
so um
i want to say a few words about the
whole human ai alliance program
so this is the this is the core team i
think it was
a project
uh awarded to professor hancock from the
psychology department at utrecht
university and also
palo's
from industrial design at oven
and supaya and i was hired to do the to
do the real work
we also have like 20 to 30 participating
researchers from all the different
institutes
they are not like very closely connected
so it really depends on
depends on whether some people want to
work with us on some of these topics
um
it's a bit about motivation so this was
project was founded in 2019 i think that
was a time
when kind of researching in interesting
ai has really gained a lot of traction
from like different disciplines
a lot of i think initiatives are like
human centered ai and in this program
the the focus is really on the tension
between
machine and human autonomy
uh whatever it is but there is a
traditional concern i think as a lot of
machines around us starts to like make
automatic decisions uh they
even move around like they can act on
their own this is of course the worry
that people might lose a sense of
control or sense of even like the core
value of like humans being like the
autonomous beings
so this was the efforts to sort of um
to strengthen the research in this area
uh
sort of
with people from both
utrecht regions and also in total
um so when i was doing the post talk
there was actually besides my research
there was a lot of effort on
creating some kind of joint uh
education so what we did was i think
something quite quite level so we
organized those events
uh inviting like students and
researchers from different institutes to
get to know each other that we founded
like seven joint master sisters project
in about two years so this was something
i really enjoy doing because you really
help some researchers to get to know
each other also students they like it a
lot because they you know they
they can work not like
completely individually but in a big
team they learn from
supervisors from different fields i
think that's really also very helpful
for for their like career
in science or in the
industry
and so talking about research uh
i want to talk about many about
two projects there were actually
many more ideas that we come up with you
know during the more last one half a
year but
in terms of research it was not optimal
for many reasons and mainly due to the
the doctor it was the project was really
in the
in the middle of the whole pandemic so
we couldn't access like nab resources
like no way to really build like
physical
prototype uh
also in terms of collaboration was also
a bit uh challenging so we
the two projects that when we talk about
uh uh both sort of online experiments
on
more kind of conceptual uh levels about
you know these autonomy issues in human
ai attraction
so in the first project
we tried to propose a new functional
model of personal autonomy
and we did three empirical studies to
test the model
we also
wanted you to know like
uh does it matter if like the the agent
that constrains you is a a real person
another person or a a ai agent
um
so this leads to the very fundamental
question what is autonomy uh
to be honest for this question
i don't feel like i can commit myself to
just one definition even even now there
were a lot of different opinions in in
the literature from different fields i
even among the
our core team they are just different
different ideas even for student
projects sometimes they got confused um
for most cases it is fine as long as you
sort of define precisely your way and
you you know you do them you focus on
your real research question
um but i do think it's very interesting
to talk a little bit about about
different perspective and how we sort of
uh
define and try to do some research on it
in our projects
and
if you don't know
much about the topic i think
the starting point for the whole
discussion about tommy it has two
different
uh traditions in philosophy so
there was
just mule according to him
autonomy is really about sort of
liberty and freedom of choice
you might have heard like that the no
harm principle so by means basically we
shouldn't
interfere with other people's choice
unless there would be some uh
unless they would hurt or harm others
so that's really the liberty perspective
and on the other hand uh you got ken's
and uh for him
autonomy means something totally
different it has a very strong moral
sense it's really autonomy really means
uh
doing the right thing or doing the sort
of the the morally
right thing and he talked about rational
self-raw rational self-control so
to give one example so uh if someone is
able to
decide all by let's say himself or
herself to to eat a lot of let's say
junk food according to uh tomiro that
would mean this person is autonomous but
not according to ken because according
to cans your actions should actually
reflect not like your sort of first
order like kind of important desire like
you know tasty food but should reflect
some kind of rational thinking you know
what was really good for you was what
was responsible for for the society for
instance
so that this is the
one key
sort of two different camps and you you
see
a lot of later definitions all follows
one of these two different different
views
and uh i think it's cycling the concept
of autonomy was mainly discussed in the
self-determination theory it's a very
popular theory so even if you know
another cycle you probably heard a
little bit about it so
according to self-determined
determination theory autonomy is like
one of the
main human needs like out of the the
three different needs
it is this need perspective means that
people kind of need autonomy in order to
do functioning in the right way or to to
have
motivation to uh to do different things
uh i'm not really a fan of this theory
since i always found a little bit awake
and even within this theory the meaning
of economy sometimes is this bit unclear
to me but
i should say the theory did contribute a
lot in terms of apparently demonstrate
the importance of autonomy for
human functioning and
well-being
um so
in daily bioacids autonomy is also a
very important
concept so here this is a very
interesting
framework called the
intervention letter
by uh
new fields
ethics council
so what they show here is they
categorize
different type of interventions in terms
of
sort of how strong they are how much
they affect people's behavior or and
also how much they
undermine people's personal autonomy so
if you look from
the bottom that move to the top so this
information becomes stronger and
stronger and also in terms of the
lactic
influence autonomy this also becomes
much stronger so for example
the most let's say
uh different one would be just you uh
observe people's behavior or immortal
people's behavior without
any active intervention then you can do
a little bit more by maybe educating
people that are moving afterwards
you have for instance you can guide
people's choice through the use of
default this links to
some research on lodging so this is
uh already like us at the middle level
then you have
if you change the incentives that's
really uh
would be have a big uh lack of impact on
personal autonomy in the end you could
basically restrict
people's choice or eliminate people's
choice altogether by maybe regulations
or by by law so that that
that would be the
strongest restriction on autonomy
and then in the
more engineering field in ai computer
science
sometimes autonomy is also used as
meaning the same like
human like capacities and intelligence
almost sometimes it use i feel like
almost
interchangeably with the word
intelligence which i think it
gives a little bit maybe
i think
like more ambiguity to what
what you want
to convey with the word autonomy
but there are more specific theories
such as
if you talk about
how to
decide whether a system is autonomous
there's one perspective that you would
look at when the systems can make
decisions and can act in the sort of
physical environment by their own so
without any
intervention from human users or
operators
uh there's another i found pretty
interesting idea
by
quite old idea in the 1990s so
according to these people they define
kind of a framework they
categorize or different entities in the
world to three different categories so
you start with like objects it's like
maybe your cops or they basically they
they just sit there pass me them
basically waiting for you to you know to
use them to to achieve your goals they
cannot actually do anything then you
have agents so you can think about maybe
software agents or some just typical
automation
so those
if you give the system kind of a goal
they will just do something by their own
but according to this framework
the agent can be called autonomous only
if
the systems would actually they can
generate their own goals
based on the external environment and
even based on the internal motivation so
[Music]
i think the agent at this level was
already qualified as kind of autonomous
according to this definition but not
according to uh and the
environment
um
i also found the idea of internal
motivation is pretty interesting because
i think it's still pretty difficult to
say what is a multi internal motivation
for a artificial system
even in today's applications
so in our research we
have the idea to
to sort of propose a functional model of
personal autonomy so
this is
actually my last idea of handcarts for
my postdoc supervisor
so according to this model
you would say a person is autonomous
if
this person has
first of all some kind of agent capacity
so that's really kind of the internal
ability like you need to have um
you need to be able to basically to to
act in the physical world it needs to be
to have
um
cognitive function to just to think that
you decide if you have you know issues
with those things then basically you
cannot have
it cannot be autonomous and then in
addition to the ability the
for a person to be autonomous
they also need to have a active goal
they need to
they need to want to achieve something
uh
for instance to satisfy their motivation
or needs and goals of course can be very
general can be very specific you can
talk about
having a goal of
living a healthy life you could also
have a goal of you want to eat salad uh
this evening so gold can
can
the level can can change from very
generous very specific
and then when there is a goal then you
can talk about the environment you can
talk about whether someone has actually
sort of opportunities to achieve the
goal by
specifying
the different uh determination so here
we have
um the different sort of
specifics about gold pursuit like the
word when and how
so in our research firstly we
focus on the determination of these
specific
goal pursuits components like the words
when and how so we assume of course we
don't look at the case when someone
doesn't have capacity or doesn't have a
even real goal so we assume that someone
has a goal and in order to pursue
a goal
these are things you need to specify so
you could call this like three different
goal pursuit components you could also
call this maybe just three different
type of decisions you need to make
before you can achieve your goal
so what does this mean so um
i think this is in a way quite intuitive
you could say the the word component is
about really
what you will do
in order to achieve a goal so this
business set like a more concrete goal
and
to basically or like kind of a behavior
that you need to carry out to achieve
the goal and i think about the when uh
components that's about ten minutes but
uh when do you want to uh
do that thing and then you have the how
component is about how
would you want to would you want to do
that and normally there are just
multiple
different methods to achieve the same at
the same goal
um
so this is like as slightly lower level
than the the component
the word component
so in the psychology literature
and also in neuroscience they are
research that shows some distinction
between for instance the world and the
world components and also the world and
how components i'm not going into
details
but here
talking about some like real-life
examples so you can think about those
different components in certain
applications
so if you look at
the subscription from ns
they have different kind of
restrictions in terms of what went on
how for instance
there you some
subscription
only allow you to for instance go to a
certain
city or place you have to sort of say
okay that's where i want to go
and other subscriptions basically
restrict you from like when can you
travel like you you can or you cannot
travel during the the peak hours
and uh finally there is also the how
aspect like you know whether you can
travel maybe in the first or second
class
uh you can also think about uh for
example uh
drivers who use uber so the uber has
algorithm that sort of manage all these
workers then their behavior can may also
be restricted
in terms of a different aspect like
which customer they
they should pick up when they should
pick up the customer and also maybe
through
which routes should they drive to the to
the customer so in a lot of real life
examples you can sort of make
distinction between these different
components so in the embryo studies we
asked the question
what is the relative importance of these
three different
decision making aspects in determining
people's
personal autonomy so
if you restrict to what when and how and
which restriction was needs to like a
stronger sense of
um
like a um
reduced autonomy
um
we have like no i think we probably we
thought like of course all three shoots
be important but the question is
bringing about the the the relative
importance and we also wanted to know
do they also interact to influence
autonomy
or the influence autonomy is more or
less sort of independent from each other
so for example if you restrict
the word
components maybe the one component
doesn't matter anymore because
maybe the world is just very strong
restricted so when you see whether there
will be any interaction or there will be
actually no injection
and secondly uh we want to know
uh does it matter if the restriction
comes from a person or from a ai agent
or algorithm
so they are of course a lot of research
looking at
differences in trust and acceptance
in terms of
decisions made by human experts or ai
algorithms and i think
sometimes
studies found that people are obesity of
reasons but in some other studies they
also found people seem to like
ai always understand that the opinion
from experts
but there's not much like really the
different type of restrictions ai
algorithms can impose on our human users
so that's that's one of the uh the novel
the
novel aspect of our study
uh so we did three studies three
experiments on experiments i want to
focus on study
1a so
because the others two experiments are
extensions and replication of the first
so in this first study
we have basically a
three by two by two by two design sounds
rather complex but the basic
uh
manipulation is that we motivate of
course the three different components so
each of this aspect could be
decided by oneself could be decided by
by another agent
uh so all these are manipulated within
subjects
and then we have
uh
another factor so the source of the
restrictions could be from a human could
be from ai agents and we have a baseline
condition which we don't specify
the type of
the source of restriction so this is
manipulated between between subjects
uh what we did was we basically give
people uh
they they
imagine themselves being different
scenarios
and
including travel
work health and social the different
type of goals they may want to achieve
and they go through like eight different
scenarios in render holders where these
different things are
manipulated and we basically
we measure uh the perceived autonomy uh
in terms of uh
the freedom of choice control
uh the restriction on autonomy and also
responsibility and
this
all these items we basically aggregate
to have one measure of a perceived
problem
so this is how a
one
the task looks like
so the first basically they read a um
kind of description of a scenario
uh for instance here is about planning
at the holiday so very brief like kind
of abstract description and also we also
basically told them like what
uh
what are the meanings of these different
decisions like what when and how
and then uh they were told that they are
going to uh
to look at eight different scenarios
and these decisions can be made either
by themselves or by
another person
and so this is how
each each trial looks like so we
basically
visualize the allocation of these
different aspects using a diagram so
in this trial then all the three
different
components are determined
by
by myself
and in this trial you see that uh this
is a case
where um
someone can decide on the when
let's say to
uh to travel but cannot decide about the
word and the how those are determined by
uh supposedly to be by by the other
person and then they answer
for the four questions for the
measurement
so
so they are
for for each um
for each type of gold pursuit there are
eight trials and this was presented in
random order
uh we also manipulate the
source of the restriction so this uh
different version of the instruction
text replacement need to read
uh again this is again quite quite
abstract we don't want to define like
what who is the other person or what
kind of ai so it's just very much on the
conception level in this
first project
and in terms of
the background we basically it's the
same we just change at the level here
for
it could be the other person in the
human condition the ai agent in the air
condition or we just basically say
something is constrained in the baseline
condition
we did two applications uh so in study
two uh
we wanted to rule out the possibility
that the
the order of introducing and
regionalizing the component would bias
the result because we always put the
word on problem when and how
but we found that actually it doesn't
doesn't matter
and
also
we want to show like can we just simply
ask people like which aspect they
consider to be the most important but
then we found what they self-report is
quite different from uh
what is revealed in the uh from the task
and uh we did a third study where we
extend uh the study to the more
organizational settings like
allocating job tasks or making like a
career plan
we also replicate the human versus ai
effects i will say you know what we
found
and
we include two additional dependent
variables like how much they would like
a decision-making situation and also um
do they actually accept sort of to go
along with with the situation defined in
the in the scenario
and so this is a
very basic result from study one so you
see
the three different between subject
conditions the baseline human ai
and
you also see
basically what we found was that
when you remove each of the the
component from one's own control
uh
pacific only would basically
go down so that's quite uh
unsurprising uh you see
uh also like what happens when persons
hear
the word class when these two are
controlled by by oneself in that trial
but not the how
and so
this is a
maybe
easier to understand the basic result
from study so here
we plotted
the
regression coefficient so
these are basically the
effect sizes of these three different
components and also the uh
the two-way interaction effect and the
three-way interaction effect
and what we found was that if you
restrict any of the three components
pacific autonomy will basically be
reduced
substantially
we found
very little interaction effect so that's
like all these different components they
uh what when how they affect autonomy
more or less like uh independence or you
see they it's actually in fact very
close to q0
we also didn't find any difference
between the human and the ai condition
that both things study one and the
stylish three so you see
uh
the bars with different colors and
really
only tiny differences
if you compare that to the
to the effect size
uh of just removing this component
that's like for instance here like
it's so close to one that means if you
uh
restrict any of those the pacific
autonomy will be reduced by that one
point on a 7.0
um we try to compare the relative
importance and in study 1 and 2 we found
sort of a consistent
order uh of what how and when
in terms of their importance so it's
like people
did consider the world to be maybe the
most central to pursuit of autonomy
followed by how and when we saw this
is pretty consistent should replicate
again but not really in study three when
we use
a little bit more concrete scenario in
organizational settings and here you
find the much smaller difference and the
same is also the when components now
becomes a bit more important than the
hub
so to conclude
uh
the three studies so i think it was not
surprising that of course in such a
abstract experiment if you just
manipulate the restriction on this
different
aspects
personal autonomy and also goal
motivation always go down
we didn't find instruction effect
meaning that
perhaps in a way you can say you can
sort of
uh the removal of one component by maybe
giving people freedom to choose maybe
another uh
in terms of another component so if you
restrict
the world component maybe it's nice to
give people to choose like when they can
do something
and we were initially kind of excited
about seems to be like a clear order in
this different aspect but that was not
really replicated in the service like
we also don't find didn't find the
difference in terms of uh restrictions
from human and ai
um
[Music]
of course at least not in such a quite
abstract experiment in real application
context of course that could still be a
different story at least i i think that
way
um
then
one thing i actually found with
struggling is like what kind of
design
implications you can think of
from the result as i said the model was
sort of really
um
found like a very strong kind of a
conceptual psychological perspective so
we also want to hear your opinion uh do
you think some of these results might be
interesting in terms of
uh creating kind of design space like
different can you separate the different
components uh like which one you want
maybe their systems to constrain and
which one you want people to be it to be
to be free to choose
so i want to continue with
somewhat different
talk about somewhat different study so
this is
um the work
oh sorry so this is the word um
mainly by my collaborator supplier from
the id departments at qe so this is a
little bit more
a little bit closer to application so we
wanted to look at what are the effects
of providing explanation and also like
maybe making people to be aware of like
this really ai or algorithm behind some
applications what would be consequences
on people's
autonomy and in terms of like very
common basic everyday sort of
interactions with ai
so
we have many three different research
questions so we want to know
does providing explanation help to sort
of protect
specific autonomy to some extent uh of
course there's a lot of research on
explainable ai so the general idea is
that this should
i think normally would be a good thing
so you want to see whether indeed if you
provide explanation maybe people are
less intimidated by some of the
automated recommendations
i want to know how does also the
uh awareness of ai influence specific
autonomy these we don't have like a
clear prediction we don't know whether
making people to be more aware of the
the algorithm or something like ais
behind the system is is beneficial
always
detrimental for for pacific autonomy
finally
we also want to explore the like
differences of course like different
everyday applications
such as like movie recommender or like
color application system kind of smart
formal application or like social media
sort of uh
filtering of like like what you read
um
so
what we did was again online experiments
where we used uh
things called design fiction method to
create different scenarios uh using some
video clips we had a 2x2 between sundeck
design
so
for for each application the participant
either they
uh they saw a explanation or they don't
so i don't see any explanation and they
were made to be
highly aware of the ai behind the system
or that was not the case
um
we got
over 300 percent from politics so about
80 percent per condition
uh so the person basically they went
through the eight applications in the
rundown order uh so we we used they did
some videos to show the behavior of
those ai infused systems i will show a
few examples of the video in a minute
then we measure
placebo autonomy
this time using a bit more
questions kind of
more relevance to the applications so
you see
for instance whether the system provides
choices based on the user's true
interest
or like the system left the user to do
things their own way
and
so we have five of these items to
imagine pacifica told me
um so this is my example
how
the solarium looks like so this is
like left legs
so she was like when you log in so this
is a condition
um
it presents of some kind of orgasm is
made
very standing so they release that you
know this
sure kind of artificial brain and you
know
and this is also a condition with
explanation uh so so for each we
recommend it there is some labels
underneath showing that
why what are the reasons that you might
like certain movies like certain actors
or certain
genre of a
film
and this is a
the
i think the the thermal starts
application
yeah so again you can see some
explanations here about why the system
set like this particular temperature for
you
and finally this is a example from the
car navigation
application
okay
so also some explanations are
here
um
so again
it was a 2x2 design so either
they see something like this like
highlighting the presence of ai or they
see
something much simpler
or and also
for some participants they
um explanations were provided by the
system or there's no explanation
so what we found
we
we checked whether the modifications
sort of
uh work or not you know whether people
actually notice the differences that
seems to be all right so um
you know people they
they were aware that
they were aware of the
explanation
so it was different rating in terms of
how much do you think the system was
providing any explanation between the
different conditions and also in terms
of the motivation of the awareness and
but
it's also it was also clear that the
meditation of this foundation worked
a lot better than the other one the
other one was maybe a bit too subtle you
know the difference between the
conditions uh was was pretty small
and if you look at
all the
results across all applications
it's actually
nothing was really much interesting what
happened uh there was no effect of
explanation knowing effect of
you know prime is like higher awareness
of ai
also no instruction effect
what becomes a little bit more
interesting is that you look at the
individual applications
and there was some somewhat surprising
to us that we found
for whatever reason uh
the car navigation system sort of
stood out
because
we seem to find actually pretty
relatively strong
effect of
providing explanation of a civil
autonomy so if they will show the
explanation they
the person they perceive the autonomy to
be higher
in in the car navigation case so you see
this visualization you can see the
distribution are
separated quite a bit
here but for all the applications there
was almost just a totally overlap
just to check was that corrected to
multiple comparisons it was so we sort
of uh divided like alpha by
eight and also to be honest we had a
quite large sample like 80 per between
subject condition
so
i was also thinking whether this is like
a flirt or
you could replicate attempts to believe
you probably would replicate but
it could be something specific really in
the design of our scenario or it could
be something more interesting really the
car navigation is uh have some like
different special attributes maybe
compared to other uh applications that's
that's i think still quite debatable
charge us to check seriously we have uh
we want to leave some room for
discussion yes
okay yeah it's almost at the end of the
talk
i have a couple more slides but i will
skip some of the more recent results
some like three five years but uh and we
also cover like the differences of
course the application so regardless of
whether
regardless of our manipulation so
providing explanation a lot or
you know priming the ai or not you also
see
[Music]
some trends in terms of like what kind
of application people
for what kind of application people seem
to be worried about maybe personal
autonomy a bit more than others uh if
you do some proper tests and you could
argue that these things like
um for social media like you know like
facebook that filter like what you see
from your friends for instance that is
you know this is like worried the most
probably by uh
the for the for the punishments also
the climate control but for like fitness
coaching and car navigation
uh those were people tends to perceive
just higher autonomy regardless of
you know the different type of different
uh manipulations
um
so i thought it was interesting to
observe the difference across the
different applications and that that
hasn't been done a lot
and the other question is why this kind
of application system stood out so is it
because of
that it's kind of a
more
have a critical application that's about
like real-time decisions or actions
uh or for or something else i
i don't know uh could also be just the
the way we design the scenario that's
then that's that's interesting um
and i think one limitation is that the
manipulation of this awareness of ai was
maybe interpreted differently by
different perspectives like showing the
brain people could think about data
privacy issues you could think about
like this maybe this system is very
smart or could could
all go
all different different ways
um i think i will stop here
there are some other things that we can
also talk maybe uh in the next
half an hour but
i want to know if there are any
questions about the uh what they're
presenting great thank you
[Applause]
um the question is to unexplain the
design scenarios implied action and so
i'm thinking maybe if one if one thinks
of the netflix
movie recommendation maybe there's an
expectation of picking one anyway so
explanations might not matter so much
ways with the nest it looked like there
was no option on the following action so
i couldn't go and suggest to change the
temperature and with the car then you
have maybe two options you can take that
electric so that was just a question if
if there was a thought around which
actions are implied by the scenario
um and then the other question was
around the
watch when um and how you spoke about
context dependency that you've seen and
to what extent do you think is also
cultural dependency
um so first so thanks for the for the
further i think very good questions
uh so for for this study i think
uh there are actually many differences
across the the scenarios to be honest uh
we try to make this to be realistic but
that would also mean
you don't have a lot of control so
uh for instance as you suggest like for
the movie
scenario uh it's like you still need to
take action you actually need to decide
but like the room temperature is sort of
it's it's that for you then you see some
expansion alone
um
those might of course
change people's experience
even though sort of
the only one that stood out was really
the calibration because for the others
sort of um
you know the motivation
we thought might influence how people
perceive autonomy was not
not really the case
and
but indeed perhaps because for the
social media and the
uh the time the temperature control
indeed people tends to
perceive indeed if you look at table a
little bit lower
just autonomy in general could indeed
maybe be because of you know in those
cases it's like the decision is made you
just see what's come out of the
uh the algorithm and uh you are either
happy or unhappy
but i guess for some other things
we recommended you still need to make
active choice uh
high navigation sort of
also a little bit
um
but i would say probably we need like
more
dedicated studies too if you want to
also separate those those different
different factors
and
in terms of the cultural difference for
the different components
um
i i don't know i tend to think if we use
such a abstract study probably not
the context
could probably indeed quite critical
and however somehow from the first two
studies we found a very consistent order
even for different sort of scenario like
planning a travel or
planning planning your work or like just
playing like a social event didn't seem
to system to matter
but then we when we switched to like
more concrete scenarios you know
organizations
then we found there were quite some
variations in terms of the relative
importance
we
so i also did a follow-up study where i
want to sort of
go beyond the the really abstract
scenario but i did the experience
sampling studies um different type of
restrictions people's decision for like
three meals like breakfast dinner lunch
and again asking like what aspect was
constrained or not constrained now over
there you again find the in the kind of
the the case of
uh dietary choices order is again not
what we found like that the caneer
what's how and when so probably it
depends a lot on different uh different
type of uh corporate suits
okay before we continue the discussion
do you mind sitting here so that we can
turn the camera so people in downline
meeting can actually see all of us yes
that's okay can you change slides from
here i think again right
[Music]
great more questions
yeah thanks so thanks for super
interesting talk
um great question actually about the
first
half perhaps so you laid out these
really nice different
perceptions of meaning of autonomy that
that they're out there
and then you propose your own one so and
i think that's the one you use here
in the setup of your experience so i
wondered
if you would have set up your experiment
differently or if you would have
perhaps do you think the results should
be interpreted differently if you had
used the different conceptions or one of
the other ones maybe the canteen one or
so would that have
yeah would have changed
um i would say it's more as follows the
more kind of the the liberty like
independence of you because still
reading about like restrictions and
choice not so much really about
sort of relational and
self-control doesn't have like a moral
aspect
um and
i i think what we
came up with like this comparison for
these three different
components is also a little bit like
investigation at slightly different
level so it's not just about restriction
or not like or what people do you know
do people follow
uh like a second order or personal
desire but it's more like
different
different uh let's say
different type of restrictions
um
so
i think in general it follows the more
perspective that autonomy is about
uh you know do you sort of
control your actions do you can you
decide something something by yourself
so that's i think that's that's how uh
how that was sort of
implemented in in the
in the experiments
um and this this distinction between the
different components i i will say it's
something something a bit quite specific
so
uh i had actually a little bit
difficult time at the beginning to see
is this really important because
intuitively
you know you can talk about those
different aspects
although in real applications they're
also open into interviewing you know
sometimes if you can can't decide like
where to go like also the timing is
probably also constrained
so um
so why i thought you know in what way
but perhaps in kind of
the design scenarios
designers can
really play with these parameters you
know to to set like you know this like
on off like to create different uh
different type of interventions
yeah so i i tend to agree with you
so my wish would also be that it really
matters which which of these
uh meanings you would use and that's why
i'm you know i'm
very interested in also why you would
think so why you chose this specific one
because
so if you if you choose perhaps your own
conception of autonomy then you might be
designing or checking or measuring for
something that you don't want to check
for but uh i think
that you
so i think in the first part of your
answer you said something that this
captures what we actually find important
in this specific context
um
is that is that the is that indeed but
is it a good interpretation what were
you just said or yeah so i think this
idea that these different components
more follow from like the some uh
research psychology about like people's
school pursuits like what are the
different aspects
different decisions
and
so
i don't know if like you take a
completely different uh
perspective and told me how would that
you know change the description of this
this very
specific things
and
so i would say it sounds like a slightly
different different angle you know it's
like about you know what what type of
restrictions rather than you know
whether you are being restricted or like
in what way you are being restricted
okay thanks
okay
uh thanks
so
you talked about it
again earlier i understand correctly in
your studies autonomy is expressed as a
number right apparently so how like so
what's the how does that measurement uh
how is that measurement defined in a
related question and it's related to
some of the things that were discussed
uh do you see some limitations in uh
reducing the meaning of autonomy
in this context to a number what might
get lost uh when we reduced
so
in i think both studies was kind of
measured on
a
uh like
one example scale like here
they basically
need
like what it stands to
feel that you have living on chores in
such a scenario so they imagine the
scenario this is this is how things are
determined they um they they basically
uh
basically choose one of the options
they think this is uh really indeed
that they have a lot of freedom when
they have legal freedom
um
we try to cover like
the concept like
some
different dimensions so we have
uh
one question about freedom of choice one
question about control
just like talking about restriction
autonomy very directly and also
responsibility that's also open
assumption that if you are
not autonomous then you are not
responsible for action
um so in this case
in the end they correlate really highly
and we aggregate it
um
a single question like reducing these
two numbers how would that sort of would
be an imitation
and
i think yes
even though i think
this is like i would say like very
common limitations in a lot of similar
studies that you try to
measure certain
people's feelings or attitudes
you know using using this type of
numeric skill you could of course
use like
a different approach uh
maybe
doing
qualitative research to ask how people
really feel then you can capture a
little bit more uh perhaps
nuances in terms of you know maybe like
again like maybe the even though what
when how
um
these different restrictions people
might have like
different specific feelings to them
that's indeed that's not really captured
by just these four items
just to quickly share so from this from
my experience with engaging with the
students in some research projects where
questions related to autonomy were
investigated
that we founded this qualitative aspect
to engaging with people and kind of
really going into the nuance that was
really useful to them by seeing what are
the design directions
uh to for a for example user user
interface
uh
so so so so that's where that nuance
came to that inspire particular things
to be designed yeah totally i think i
think also
some measures should be on the behavior
level not just
the waiting but you know how they you
can also sort of maybe observe how they
would continue to maybe interact with
system i think uh that would be
indeed also very useful um
studying like that alternative
automation
[Music]
we are running out of time
so maybe more questions from there
yes if possible
thanks uh
no questions from your
audience uh there is a question from
timon but he's left already so i think
we just
reflect him later on so my question was
if i take a step back from your results
it seems like
there were no no big effects
uh and so similarly
my sense is also that maybe a
qualitative perspective
would actually really enrich the
understanding but i was also thinking
i'm a quantitative guy after all as well
so i think maybe there's big confounding
factors
or maybe it's it's not a good metric
that but also people maybe it's bigger
so i was thinking
where are the consequences
are the consequences of you know
uh in all of these these cases of what
you would go wrong
and
what's your ability to resolve
misalignments
and my senses because that's what this
community is about this or actually
having some kind of control and so it's
also related to one of these autonomy
factors and so my question is could you
respond to what i think you think that
that is
maybe you know preconceived perceptions
of consequences and ability to actually
do something about it
actually played part in the in the
variability
and and therefore perhaps the lack of
big effects yes it's a really good point
because i also think like one may
limitation i wouldn't say it's like
really confined result but
in all these cases it's not like very
abstract scenarios
they don't really make decisions they
they give a scenario how the decisions
will play out so that's the only thing
they perceive then they raise basically
how they feel about that situation so
they don't make a real
decision they don't make choices
there's like no sort of consequences
like manipulate in the experiment so um
if they just look at you know how this
is supposed to be the case in
show showed in in those diagrams uh i
think different people can relate to to
those scenarios in different ways and
think about what would be consequences
uh i think even
the way we describe like the other agent
is also very abstract there's no like
you know what is the relationship with
this person to you or it's just not it's
not like
uh so the these all these things are not
let's say uh flesh now
um
so
um so one my idea i have sort of at the
end of those was we need to
you can still try to see
how you know does these different
components matter you know what the one
how
about for instance using
a different approach where you have
people actually making decisions
together with the system like maybe
maybe maybe a chatbot you know some kind
of conversation and you see different
options like some constraints someone
not constrained i think in that case i
would say
it
should give you a bit more realistic
tests on how this how this difference uh
passes different dimensions
okay great um
yeah i'd say uh
it's technically over but we can stick
around for uh
or enough to party uh because i think
the room
is i mean we didn't book it after two
but nobody's showing up sorry for half
an hour yeah great uh but with that i
think we have to say goodbye to our
online participants
and then stop the recording
and then i think
oh okay
you're just yeah
to explore


GPT 4 Can Improve Itself - (ft. Reflexion, HuggingGPT, Bard Upgrade and much more)

gpt4 can improve Itself by reflecting on
its mistakes and learning from them even
if the world does pause AI development
gpt4 will keep getting smarter drawing
upon the stunning reflection paper and
three other papers released only in the
last 72 hours I will show you not only
how gpt4 is breaking its own records but
also how it's helping AI researchers to
develop better models I will also cover
the groundbreaking hugging GPT model
which like a centralized brain can draw
upon thousands of other AI models to
combine tasks like text the image text
the video and question answering the
reflection paper and follow-up sub stack
post that caught Global attention was
released only a week ago and yes I did
read both but I also reached out to the
lead author Noah shin and discussed
their significance at length others
picked up on the results with the
legendary Andre carpathy of Tesla and
openai fame saying that this
metacognition strategy revealed that we
haven't yet seen the max capacity of
gpt4 yet so what exactly was found here
is the headline result I'm going to
explain and demonstrate what was tested
in a moment but look how they used gpt4
itself to beat past gpt4 standards using
this reflection technique this isn't any
random challenge this is human eval a
coding test designed by the most senior
AI researchers just two years ago the
designers included Ilya sutskovar of
openai Fame and Dario amade who went on
to found anthropic these are realistic
handwritten programming tasks that
assess language comprehension reasoning
algorithms and Mathematics so how
exactly did gpt4 improve itself and beat
its own record because remember in the
distant past of two weeks ago in the
gpt4 technical report it scored 67 not
88 well here is an example from page 9
of the reflection paper as you can read
in the caption this was a Hotpot QA
trial designed specifically such that
models needed to find multiple documents
and analyze the data in each of them to
come up with the correct answer notice
how initially a mistake was made on the
left by the model and then the model at
the bottom reflected on how it had gone
wrong in a self-contained Loop it then
came up with a better strategy and got
it right and the authors put it like
this we hypothesized that llm's large
language models possess an emergent
property of self-reflection meaning that
earlier models couldn't do this or
couldn't do it as well it's a bit like
GPT models are learning how to learn in
case you think it was a model blindly
trying again and again until it was
successful no it wasn't this was another
challenge called Alf world and look at
the difference between success without
reflection and success with the
reflection I discussed this of course
with the lead author and the goal was to
distinguish learning curves from
self-improvement to simple probabilistic
success over time if you're wondering
about Alf World by the way it's about
interactively aligning text and embodied
worlds for example in a simulated
environment the model had the task of
putting a pan on the dining table and it
had to understand and action that prompt
so as you can see this ability to
reflect doesn't just help with coding it
helps with a variety of tasks at this
point I want to quickly mention
something I know that there will be a
couple of well-versed insiders who say
didn't gpt4 actually get 82 percent in
human eval in the Sparks of AGI paper of
course I did a video on that paper too
and asked the author of reflection about
this point there are a few possibilities
such as prompting changes and the
sparked authors having access to the raw
gpt4 model but either way it is the
relative performance gain that matters
whichever bass line you start with gpt4
can improve on it with a reflection and
the 88 figure is not a cap the author
has observed results in the last few
hours as high as 91 percent but before I
go on I can't resist showing you the
examples I found through experimentation
and also shared with the author take
this prompt that I gave gpt4 write a
poem in which every word begins with e
now as you can see it did a good job but
it didn't fully get it right look at the
word Ascent for example without
mentioning anything specific I just then
wrote did the poem meet the assignment
not even a particularly leading question
because of course it could have just
said yes gpt4 then said apologies it
appears the poem I provided did not meet
the assignment requirements not every
word begins with the letter e here is a
revised poem with every word beginning
with the letter e remember I didn't help
it at all and look at the results every
word begins with e how far can we take
this for the next example I chose
mathematics and asked write me a five
question multiple choice quiz to test my
knowledge of probability with correct
answers and explanations at the bottom
there should only be one correct answer
per question it comes up with a D decent
quiz but notice a problem in question
three for example the probability of
drawing an ace or a king is indeed 8 out
of 52 but that simplifies down to 2 out
of 13. so two of the answers are correct
and I explicitly asked for it not to do
this in the prompt so can the model
self-reflect with mathematics kind of
almost look what happens first I give a
vague response saying did the quiz meet
the assignment GPT 4 fumbles this and
says yes the quiz did meet the
assignment hmm so I tried did the quiz
meet all of the requirements and gbc4
says yes so I did have to help it a bit
and said did the quiz meet the
requirement that there should only be
one correct answer per question that was
just enough to get gpt4 to self-reflect
properly and it corrected the mistake I
must say it didn't self-correct
perfectly notice it identified C and D
as being correct and equivalent when it
was B and D but despite making that
mistake it was able to correct the quiz
in case you're wondering the original
chat TPT or gbt 3.5 can't self-reflect
as well I went back to the perm example
and Not only was the poem generated full
of words that didn't begin with e also
the self-reflection was lacking I said
did the poem meet the assignment and it
said yes the poem meets the assignment
as the lead author Noah Shin put it with
gpt4 we are shifting the accuracy
bottleneck from correct syntactic and
semantic generation to correct syntactic
and semantic test generation in other
words if a model can know how to test
its outputs accurately that might be
enough even if its initial Generations
don't work it just needs to be smart
enough to know where it went wrong
others are discovering similar
breakthroughs this paper from just three
days ago comes up with this
self-improvement technique they get gpt4
to frame its dialogue as a discussion
between two agent types A researcher and
a decider a bit like a split personality
one identifying crucial problem
components and the other one deciding
how to integrate that information here
is an example with Gypsy 4's initial
medical care plan being insufficient in
crucial regards the model then talks to
itself as a researcher and as a decider
and then lo and behold it comes up with
a better final care plan the points in
bold were added by gpt4 to its initial
care plan after discussions with itself
and the results are incredible
Physicians chose the final summary
produced by this dearer dialogue over
the initial Gypsy 4 generator summary 90
to 10 that's the dark red versus the
Pink I'm colorblind but even I can see
there's a pretty big difference the
authors also introduce hallucinations at
different levels low medium and high and
they wanted to see whether this dialogue
model would reduce those hallucinations
these are different medical gradings and
you can see that pretty much every time
it did improve it quite drama
automatically and then there was this
paper also released less than 72 hours
ago they also get a model to recursively
criticize and improve its own output and
find that this process of reflection
outperforms Chain of Thought prompting
they tested their model on Mini wob Plus
plus which is a challenging Suite of web
browser-based tasks for computer control
ranging from simple button clicking to
complex form filling here it is deleting
files clicking on like buttons and
switching between tabs a bit like my
earlier experiments they gave it a math
problem and said review your previous
answer and find problems with your
answer this was a slightly more leading
response but it worked they then said
based on the problems you found improve
your answer and then the model got it
right even if you take nothing else from
this video just deploying this technique
will massively improve your outputs from
gbt4 but we can go much further which is
what the rest of the video is about
before I move on though I found it very
interesting that the authors say that
this technique can be viewed as using
the llm's output to write to an external
memory which is later retrieved to
choose an action going back to carpathy
remember that this critique retry
metacognition strategy isn't the only
way that gpt4 will beat its own records
the use of tools as he says will also be
critical less than 72 hours ago this
paper was released and arguably it is as
significant as the reflection paper it's
called hugging GPT and as the authors
put it it achieves impressive results in
language Vision speech and other
challenging tasks which paves a new way
towards AGI essentially what the paper
did is it used language as an interface
to connect numerous AI models for
solving complicated AI tasks it's a
little bit like a brain deciding which
muscle to use to complete an action take
this Example The Prompt was can you
describe what this picture depicts and
count how many objects in the picture
the model which was actually chatbt not
even gpt4 or use two different tools to
execute the task one model to describe
the image and one model to count the
objects within it and if you didn't
think that was impressive what about six
different models so the task was this
please generate an image where a girl is
reading a book and her pose is the same
as the boy in the image given then
please describe the new image with your
voice the Central Language model or
brain which was chattybt had to delegate
appropriately all of these models by the
way are freely available on hugging face
the first model was used to analyze the
pose of the boy the next one was to
transpose that into an image then
generate an image detect an object in
that image break that down into text and
then turn that text into speech it did
all of this and notice how the girl is
in the same pose as the boy same head
position and arm position and then as a
cherry on top the model read out loud
what it had accomplished this example
actually comes from another paper
released four days ago called task
Matrix remember how the original tool
former paper used only five apis this
paper proposes that we could soon use
millions in this example the model is
calling different apis to answer
questions about the image caption the
image and do out painting from the image
extending it from a simple single flower
to this 4K image going back to hugging
GPT we can see how it deciphers these
inscrutable invoices and reads them out
loud and can even perform text to video
with an astronaut walking in Space at
this point I can't resist showing you
what CGI video editing might soon be
possible with AI here's Wonder Studio
which is backed by Steven Spielberg
welcome to wonder Studio we're making
movies with CGI is as simple as
selecting your actor and assigning a
character
the system uses AI to track the actor's
performance across cuts and
automatically animates lights and
composes the CG character directly into
the scene
[Music]
whether it's one shot or a full sequence
Wonder Studio analyzes and captures
everything from body motion
lighting compositing
camera motion
and it even tracks the actor's facial
performance
these advancements do seem to be
accelerating and requiring fewer and
fewer humans this paper showed back in
the before times of October that models
didn't need carefully labeled human data
sets and could generate their own going
back to the language models can solve
computer task paper the authors seem to
concur they said that previously
significant amounts of expert
demonstration data are still required to
fine-tune large language models on the
contrary the agent we suggest needs less
than two demonstrations per task on
average and doesn't necessitate any fine
tuning this reminded me of the alpaca
model that fine-tuned its answers based
on the outputs of another language model
human experts were needed briefly at the
start but far less than before a bit
like a child no longer needing a parent
except maybe gpt4 is on growth steroids
Ilya satsgiver from openai put it like
this I mean already mostly data for
enforcement loan is coming from AIS the
humans are being used to train the
reward function but then the but then
the reward function
enter and in its interaction with the
model is automatic and all the data
that's generated in the during the
process of reinforcement learning it's
created by AI before I end I should
point out that these recursive
self-improvements are not limited to
algorithms and apis even Hardware is
advancing more rapidly due to AI this
week we had this from Reuters Nvidia on
Monday showed new research that explains
how AI can be used to improve chip
design by the way this includes the new
h100 GPU they say that the Nvidia
research took reinforcement learning and
added a second layer of AI on top of it
to get even better results and to go
back to where we started the gpt4
technical report showed that even with
compute alone not self-learning we can
predict with a high degree of
specificity the future performance of
models like gpc5 on tasks such as human
eval these accelerations of AI are even
giving the CEO of Google Whiplash and I
can't help feeling that there is one
more feedback loop to point out as one
company like openai make breakthroughs
it puts pressure on other companies like
Google to catch up apparently Bard which
has been powered by Lambda will soon be
upgraded to the more powerful model Palm
with self-improvement tool use Hardware
advances and now commercial pressure it
is hard to see how AI will slow down and
of course as always I will be here to
discuss it all thank you for watching to
the end and have a wonderful day