David Roodman - Economic history and the road to the singularity

hey everyone jeremy here welcome back to
the tour of data science podcast and
today we're talking about two
fascinating questions that don't seem
related at all
but actually are the first one is very
simple
if you had a hundred dollars and you
wanted to use it to create as much
joy prosperity and well-being in the
world as possible
what would you spend it on the second
has to do with our long-term future as a
species
are we on the brink of an ai-powered
economic revolution
and if so what can we learn about the
history of human economic activity up
until now
that we might use to help us navigate
that transition when it happens
so today i'll be exploring those
questions with my guest david rudman
who's a senior advisor for open
philanthropy because of his role at open
philanthropy david spends his time
exploring questions that are as big as
they are practical
these are things like does tough on
crime legislation do more harm than good
or can we meaningfully reduce poverty by
using strategies like microcredit
and as we'll see he's also done a lot of
interesting thinking on predicting when
advanced ai systems might overtake human
beings and our reasoning abilities
the conversation was just a ton of fun
and it connected so many different ideas
that i hadn't linked together in my own
mind before
i hope it does the same for you i hope
you really enjoy it and without further
ado i'll step out of the way and
let it start well david thanks so much
for joining me for the podcast
uh it's a pleasure to be here i'm really
excited to have you i discovered you
through
not through your book actually on um on
micro credit which is really
really excellent really cool but rather
through a post that you wrote
about sort of long-termism and
long-termism view through the lens of ai
and i really want to dive into that i
think i want to dive in as well into a
number of explorations you've done sort
of
re-examining um some some earlier
results that people had put together in
the sort of philanthropy space
about what initiatives are worth
investing in and which may not be
um so we've got a lot to talk about but
before we get to that i'd like to
situate things a little bit
and talk about your backgrounds for what
brought you to this space of
altruism and to open philanthropy which
is where you're working out
oh um well i'm old enough that that
could take an hour to tell you all that
but i'll try to avoid that
uh i am a child of divorce and in some
ways i think my life has been about
trying to fuse the aspects
of my personality that you know my
parents gave me or inculcated in me over
the course of my life so my dad
introduced me to computers when i was 12
years old
you know i learned my first programming
was on punch cards on an ibm mainframe
and so that's been a part of my life and
i've always been interested in coding in
mathematics
but my mother was also someone who came
in early adulthood really was part of
the
feminist movement of the 1970s and she
you know gave us both my sister and i a
strong sense of
our responsibilities as citizens you
know and i've always been trying to
figure out how to make
how to combine my interest in public
policy and public welfare with my
tendency to code and think about things
mathematically
and so in retrospect it can all seem
like it was obvious but i didn't
know how it was going to work out but i
spent a lot of time working for think
tanks
in washington dc which is where i live
those are generally you know found
funded by foundations philanthropies
in order to do analysis of complicated
problems that's relevant for actual
decision making so i've learned about a
lot of different topics over the years
through that kind of work interesting
yeah and
one of the things i find particularly
intriguing about open philanthropy
is it is sort of focused or embedded in
this matrix of effective altruism sort
of effective altruist community
and one through line there really is
looking critically at
the kinds of problems that we invest
money in solving
and asking you know is this really the
best place for our money to go
that's led to some pretty interesting
counter-intuitive decisions in terms of
like
what does get funded would you mind kind
of providing a bit of an overview like
what are some of the more maybe
surprising things
that you think are worth funding that
people might not realize
so yeah i work at a place called open
philanthropy which
is based in san francisco it's
essentially a foundation although it
doesn't actually hold a lot of assets
rather it makes recommendations to make
grants
and most of the money that we're
influencing now comes from
dustin moskowitz and his wife carrie
tuna dustin was one of the founders of
facebook
and i've been there four or five years
and you're absolutely right it very much
is influenced by the philosophy or the
way of thinking that's called
effective altruism which is about
kind of like i was just describing in a
different way you know about
myself it's about trying to connect um a
commitment to making the world a better
place
with tough-minded thinking about what
actually works and what's most important
and uh through process that i don't want
to take much credit for i have a great
admiration for but wasn't closely
involved with
we have isolated maybe three or four
major cause areas and some
minor ones and we're adding to that list
over time and some of them interestingly
seemed kind of weirder and crazier when
they were first
chosen four or five years ago than they
do now so
we were working on pandemic preparedness
when
that took a lot of explaining right uh
obviously our grants did not stop the
pandemic but hopefully we've made we're
making some difference in the response
and in preparation for future ones
and also uh we've been made important
grant makers
in criminal justice reform in the united
states
trying to reduce the number of people in
prison because per capita the united
states
is by far the highest in the world and
incarceration with the possible
exception of north korea which is
you know not a flattering comparison
so we've been working hard on that and
that has gained a lot of steam in the
last few years since we started working
on it
another one for us is farm animal
welfare which can sound a little funny
but you know there are billions of farm
animals
suffering right now all the time and
if you even say that you know a
chicken's life counts one percent as
much as a person's life
and you do the math um that's a lot of
suffering compared to human suffering
and if uh with uh campaigns to get
corporations to commit to changing how
they farm
uh can make big differences for billions
of animals
with only millions of dollars that's
pretty good bang for the buck
so those are some of some of the areas
that we're working on another one which
of course we'll talk about
is and this also is something that seems
less crazy than it did say five years
ago
is ai safety make trying to do
what we can to make artificial
intelligence safe for society so
it's more source of benefit than harm i
actually
what i find really interesting here is
there seems to be thematic interplay
between
all those initiatives in kind of subtle
ways that might not be obvious
not the least of which is the way i
often feel personally
when i talk to people about ai safety is
a lot like
a person in like january 2020 must have
felt
if they were talking about the
coronavirus where you have this
exponential process
or as you've pointed out super
exponential process that's unfolding
that
has the potential to take us all by
surprise you can see where the math
leads if you just
carry it forward iteratively over time
and yet
our everyday lives really don't seem to
reflect that underlying reality
is this um is this something that you've
gotten good at at sort of conveying or
arguing like like how do you how do you
get people to understand sort of more
abstract risks
and get them to take them seriously
that's a great question
and it's really one of communication i
don't know if i'm especially good at it
i'm
you know i'm someone who likes to get
immersed in a set of ideas and then
explain those ideas as well as i
i can sort of like a teacher but if
you're not a motivated student
you may still not buy it
[Music]
so i'm not sure i have much intelligence
to say except that you need to do things
like
make your arguments really concrete draw
on metaphors connect to people's life
experiences
you know we can talk much more
convincingly now about preparing for the
next pandemic
having been through one because that's
an easy example yeah
i should say that you know what i just
listed a bunch of cause areas and they
do seem like a kind of a grab bag
ai safety farm animal welfare but they
come out of a process
some of whose virtues i think you've
just illustrated that is to say
we were able to arrive at priorities
even though there was a certain
failure of imagination that makes it
difficult to see why those are important
and we did that partly by being
systematic asking lots of people what
they thought was important and then
reflecting on that then we had some
filters you know looking at
looking for things that have potentially
very large impacts that are important
that can infect millions or billions of
people or animals
looking for things that are neglected so
that's part of why
it's a bit of a grab bag because we're
doing we're looking for stuff that
most other foundations are not paying
attention to so it doesn't mean that
these
we think these are the only important
things just where we think we can do the
most good
yeah that that's all struck me as one of
the more difficult um
from a human level relatability
standpoint one of the more difficult
aspects of
adopting this kind of of rational
position is like
you have to unfortunately be the one
telling people like yeah that you know
that
metropolitan art grant or or that that
um
whatever investment in even education in
our
communities or something like that is
great but
this dollar could probably the marginal
dollar could be spent better
you know doing this this deworming
campaign or focusing on ai safety or
whatever it is
um that kind of seems like a there's a
risk there right because you can put
people off i suppose if you're not
careful yeah
right and to so to extend
we are willing to embrace that for
example we're willing to say that we
really think that there are some
charities
that uh are doing more good than others
or have better evidence that they're
doing good
that's an important conversation to
provoke at the same time i think it's
important to recognize that all of us
you know spread our money among
different things that arrange from
rather selfish like when we buy coffee
for ourselves um
to very selfless and you know giving for
example giving to your local your
children's school is sort of somewhere
in between there
right yeah and we're not too interested
in telling people how to make that
allocation we just want to say okay for
the part of your spending that you want
to spend in the most altruistic way
here's our advice or here's what we're
thinking about doing
interesting yeah that makes perfect
sense um and it's funny the moment
you talked about you know that that
spectrum between selfishness and
altruism
i sort of start to see a lot of my
day-to-day choices that i might consider
altruistic reflected in that it's like
you know you're
you might be mentoring one person but
you really get the reward of seeing the
smile on their face you're the one who
gets to consume that benefit
um it's not like you're yeah spending
your time investing in something
abstract
um now speaking of abstractions so i
guess one skill we talked about
the necessity of arguing for these
abstract risks things that are worth
funding that
might not be reflected in our day-to-day
experience of the world one of the
the areas where that's a really big
issue is ai safety
and i really loved like i read your your
piece
um i was gonna say cover to cover but it
was it was a nice hefty blog post about
sort of what we're looking at from the
standpoint of human economic history how
that ties into ai
um without being too specific about
where to start this conversation i mean
i'd love to just hear you
lay out what the the baseline level that
argument looks like and maybe we can
poke at it and explore it
yeah so as i said we are making grants
that are meant to
get more people thinking and working on
uh how to make artificial intelligence
safe
uh now because we are a fairly
reflective
organization and one that values
internal debate
even as that decision has been made um
[Music]
the people who run the foundation
notably my boss holden karnovsky
have invited other people to challenge
that thinking
or at least to sharpen our thinking
about it and so one of the key questions
that's really hard to nail down is
how likely is it that artificial
intelligence could get to the point
where it equals or surpasses human
intelligence
and when might that happen because if we
were somehow to be able
to come to the conclusion that it's not
going to happen or it won't happen for
300 years
then that would call into question the
value of our grant making so it's in
this that it's in that spirit that i was
asked to
look at one particular angle on this
question of the timing and
likelihood of an artificial intelligence
breakthrough
there are different angles one can look
at how much computation we think goes on
in the brain and then asks
when will it become affordable to make a
computer do that much compute and so on
um the particular angle i was encouraged
to look at was
looking at the history over like 12 000
years or even
2 million years of economic output
or the scale of the human system so
i guess really there's two main ways we
can look at the scale of the human
system one is the number of people
and the other is their economic output
what we now call gross world
product and one of the really strange
facts that emerges when you look
really long term not over a century but
over millennia
is that the larger the human system has
grown
the faster its growth rate and i don't
even mean exponential growth
which is what we have like with the
virus sometimes i don't mean that it's
grown at three percent a year and so
it's gotten bigger and bigger
i mean like it used to grow at point one
percent a year and then it grew at one
percent a year
and maybe someday it'll grow at 10 a
year or 100
a year and so the question is if we
look at the the quant if we quantify the
past
does that tell us give us any insight
into what the future might look
like because it seems like if that were
to give us
more confidence of a of another
economic explosion like the industrial
revolution
it seems like the only way that could
happen at least the most plausible way
that could happen
is with some kind of big artificial
intelligence breakthrough
one of the really interesting
mathematical features and i mean i'll
even call it alarming mathematical
features of what you laid out there
really was and i think it slips by you
so easily when you say it but
the idea that time actually starts to
effectively compress with human progress
and it's almost like i think you
mentioned at one point in the uh in the
write-up it's like
you know you're um uh
you're gonna hit infinity in finite time
you know it's like it's like one thing
to say
we will hit an arbitrary level of of
economic output at some point at any you
know if you project out in the future
yeah at some point we'll hit a global
gdp of like um
or gwp that is of however many trillion
but to say no we're going to hit that in
finite time by tuesday next week we're
going to hit that point
um is is a much more dramatic statement
like what to you what does that that
event
imply is is there something like in your
model of the world like
do you think that that means i i don't
know what that could mean actually
do you have any thoughts about that yeah
so i mean i think you put it very well
and a point that i open the blog post
with is this
strange fact that you know if you plot
the historical data series
on gross world products so you know back
in 10000 bc
we very roughly estimate that it was
like 1.6 billion dollars
and today it's i don't know what it was
70 or 80 trillion
massive increase over that time anyway
if you plot the data points
in a way so that they look kind of like
i guess i should do it this way
i don't know well it looks sort of like
a straight line which means
both axes are logarithmic um and then
extended forward
uh it predicts that you'll get to
infinite output in the case
in my best model fit you get the
infinite output by 2047.
and i don't take that seriously right i
don't think we're going to have infinite
output by 20
and 20 by 2047. but i do feel like
you can't you don't there's there's
something to learn
there like it's not optimal to just say
oh that's stupid and move to the next
blog post
yeah but i feel like we have to ask the
question why is it
that the simple model that best
describes history predicts an impossible
future
what should we make of that and i guess
to the extent that this model
like has been very useful in terms of
description would have been
very useful in describing the past two
centuries of economic
output despite quite a few bumps in the
road um that seemed to iron out over
time just to kind of
fit this model again like
to suggest that okay now all of a sudden
this you know 2047 it's just 27 years
from now
like that starts to really sharpen um
maybe the implications of this
are there other other economic models
that have like that have broken down in
this way in the past
that that have sort of that we can think
about in retrospect and study
when you say broken down you mean have
ended up being wrong or have predicted
these kinds of strange
changes yeah i guess i'm thinking like
other areas where infiniti's might pop
up because i think your analysis was
like
you know an infinity happens so
something weird is going to happen there
i'm curious about whether other economic
models might have predicted infinities
in the past and
what are the sorts of things that happen
in those regimes
yeah it's a good question the basic
observation that i just made
is not new with me i think probably and
i talk about this in the post
first time it appeared really was in
1960
when it was done just with population
starting from the year
zero and again the same point was made
that the authors
predicted that we would have an infinite
number of people on the earth
by 2026 november 20 number
friday 13th and 2026. so that
extrapolation is new and um somewhat
distinctly
in the world of economic modeling
modeling i should say macroeconomic
modeling which is where you try to
understand the dynamics of an entire
economy not a single factory
uh that that's a that's a discipline
that really gains steam starting the
1950s
they have often struggled uh with this
question that
very natural models that you can write
down that seem to fit the present
tend to explode and so they have worked
sometimes with more contortion than
others
in order to avoid such strange
implications perhaps out of fear of not
being taken seriously and not getting
published
i should say that in the last two
centuries
it's not clear that the the simple model
we've been talking about
works as well that there was a big
you know growth take off around the
industrial revolution
and but since then growth in the
industrial countries has
better been better approximated by an
exponential model
where the growth rate each year is
fairly steady you know maybe three
percent a year
rather than increasing um and so that
could mean
that already the model i've been talking
about where growth
is accelerating is is archaic
and is a thing of the past or it could
just be like we're in you know
just a little random ripple and a longer
term trend and you know it's hard to
know
and do you think that there's a a kind
of
an understandable reason why the model
takes this shape like
when we talk about time compression the
way i've always conceptualized it when
i've heard arguments like this is it's
something like
you know one day some hominid in in a in
a cave
comes up with like the idea of fire and
then fire
allows us to do a task that might have
taken us a month
in two weeks so effectively time is then
compressed
and so we play this forward you know
thomas edison comes up with a light bulb
and all of a sudden tasks that would
have taken us a year take us six months
and so on and so forth so
it's effectively like there's a kind of
compression but i feel like that
model if anything that model should make
the last 200 years follow that trend
even more like
it seems like we've had we've been
better at coming up with new ideas and
distributing them
than we had in the past so i guess i'm i
i'm sort of looking
at this logical juncture where it's not
clear to me which of my assumptions are
failing
in that respect do you have any thoughts
about that no that's that's well put
um uh so what's interesting about
this this model we keep talking about of
super exponential growth of accelerating
growth it's not just that it's a simple
mathematical pattern that fits the
history but that there is an intuitive
theory that you've just described that
helps
us understand why growth would
accelerate over time and that is
technology
it only takes one person to invent fire
and then that idea can spread uh
and only takes one person to invent
semiconductors and the idea can spread
and so the more people there are and the
more resources they have
to and to you know engage in research
then the faster society should be able
to generate new ideas which then means
more productivity which means more
ability to
generate ideas so it seems like they're
there that
is something important dynamic that has
occurred over the course of history is
acceleration in their pace of innovation
which then allows more acceleration
so the counter-veiling question is and
yes we have more capacity for innovation
in the world today than ever before
counter-veiling idea and there's a lot
of serious argument
for this idea evidence and theory is
that maybe we're running out of things
to discover
even though it seems like things are
changing so fast there are a lot of ways
in which life has not changed
for people in industrial countries much
over the last
50 years you know the 747 has been in
service for 50 years
we haven't really improved that much on
it except in some energy efficiency
respects
uh and so you know and if you look at
the rate of innovation for example in
physics it doesn't seem
like we're having the same breakthroughs
today that they had a hundred years ago
so may and and well and those
kinds of discoveries basic things about
quantum mechanics
really do translate into
useful technology and so when we stop
making basic science discoveries that
may eventually mean we stop making
important technological discoveries
uh and so this is a really important
question to ponder and i don't have any
easy answers to it i certainly don't
dismiss the point of view
that we are running out of things to
invent the big question the big possible
exception does
seem to be artificial intelligence if
you can get machines
to do basically if we can do for the
brain
what we've already done for most of the
rest of the body you know we've already
got machines that are capable of
locomotion right
and that's been a major source of change
in human affairs
if we can do the same thing for brains
that just seems like that could have
radical implications yeah this is
very much striking a chord in a context
where i've been knee-deep and like
peter thiel talking about the great
great stagnation and
just this idea of atoms versus bits and
and that often does seem to be the sort
of dichotomy
that people draw they'll say like you
know we've made great progress in the
world of bits
since the the 50s computers and
simulations
and vr and iphones but but in the world
of of atoms like we've
you know medical treatment seems to be
slower education's more expensive
um you can almost just see these things
plotted when you look at the comparative
rates of inflation
for products like housing and medicine
versus products like electronics which
have just been deflating like crazy um
it's
i guess one of the things that your
analysis makes me wonder about is like
the extent to which there's actually any
meaningful distinction though between
those two worlds
in the long run it seems as if perhaps
we need to make a certain amount of
progress in the world of bits
because we'd done all we could with like
physical meat bodies
um you know working on our own with our
limited cognition there's a sense in
which we need to outsource some of that
compute to bits
and that's like the rate limiting step
once we unblock that
perhaps things are going to change
radically would that be a
like a fair encapsulation of the thesis
um
yeah of my thesis yes sir the yeah the
growth thesis i guess the idea i think
yes i think that that would be a fair
encapsulation of the
the growth view um
a couple of thoughts on that one is you
know i think it's it is possible that
there are major domains
of the economy and of our lives that are
just not going to be susceptible to that
much
innovation in bits like like you know
machines for moving us around can only
become so efficient we can't
have 100x improvement there uh this is
one example
uh i'm personally interested
in this question less because i really
want to know whether economic growth is
going to go to forty percent a year
right compared to like three percent now
i'm less concerned about that
than whether
the models and the math are signaling to
us the plausibility of some radical
change
just you know just some radical economic
change which might not
change every sector of our economy in
our lives but could still
be really important as important
economically as the industrial
revolution or at least on that order
and when you say radicals change i mean
i guess that that sounds
exciting but it also sounds ominous and
you've alluded to ai safety so i'd love
to tie these two together
what is it that you think so first off
um
what's your sense as to what could go
wrong and then what's your sense as to
what we could be doing today to mitigate
that
yeah this is an area where i feel like
i've read
more than most people but i don't feel
like an expert haven't written and
spoken much about it so
i feel like i'm mostly you know citing
the thinking of others
um you know there's some talk today
around the ethics of ai and the societal
effects of ai that focuses on whether
we may be training in biases into our
systems things like that
um i think we we see that as important
at open a
at open philanthropy but we're more
concerned about the things that could
be more more radical things that could
change you know how
how our large parts of the economy work
uh the the fear
to accentuate the negative is that if
you get agents
that are capable of functioning in the
physical world or the virtual world
world as effectively as human beings
and by agent i mean you know a soft
combination of software
and hardware
but they're running on silicon rather
than on protein which is what our brains
run on
that if they can get as good as us
there's no reason to think that they
couldn't become
10 times as fast as us a week later or a
year later
or 100 times faster and that they
couldn't multiply much more rapidly than
humans can
and then there's just massive scope for
unintended consequences
right not not maybe maybe the
consequences would be intended by the
ais but they wouldn't be intended by us
um most uh
artificial intelligence agents are
defined to optimize on certain goals
and so uh i think it was eliezer
yudkowski who who
first disseminated the idea of being
paper clipped the paper clipping idea
the idea that
we could create an agent that was
instructed to make as many paper clips
as possible
and then it proceeds to turn the entire
planet into paper clips because no one
can stop it
so the point is not that an artificial
intelligence would would view us
malevolently but it just wouldn't care
about us
and would be credibly capable
by human standards of achieving whatever
goal it was programmed to achieve
and so that's the kind of thing that
we're worried about in the extreme
presumably there's a spectrum of
possibilities
and even if the extreme doesn't isn't
realistic
it's hard to judge um there may that it
may be a stand-in
for less extreme things that we can do
something to prepare for by
for example creating a culture of safety
in artificial intelligence research the
way we do in drug research
and lots of other fields it's i find it
really interesting
the emphasis on the culture of safety
it's definitely something that
that resonates with me because well not
least because it seems as though there
are some
technical problems with making these
long-term future ai systems safe
that we probably aren't even aware of
today like
there's a limited degree to which we can
actually solve technical problems
perhaps that
will be relevant at that stage in the
game there are surely technical problems
we can be working on today as well
but it seems like this is almost a part
kind of technical challenge for today
and then a part cultural kind of
embedding the right values so that
by the time we we get to the stage where
these kinds of machines are
on on the cusp of verizon we at least
have some kind of coherence like people
are aware of safety
they're aware of the paper clip type
risk involved in this
is there is there a division of emphasis
at open fill
on whether to focus on culture versus
safety are both
being looked at or oh um
you know i'm not i don't know our
programming or our grant making
intimately
i'm not sure that that distinction would
stand up because at the end of the day
we're giving grants to try to support
people and thinking about certain things
rather than thinking about other things
writing about certain things and talking
about certain things rather than other
things with their limited time
and at some level that just is culture
change you know
yeah that's true i guess you get a bunch
of people working on technical problems
today that kind of seeds the technical
research groups that end up
becoming yeah makes sense um yeah and
now in terms of
the paper clip bit like one of the
things i always find interesting when i
have conversations with people about
this is like the you know the paper clip
seems like a really extreme example
you'll tell them like you make an ai and
you tell it optimize for paper clips it
realizes there's like
there's iron in the ground there's iron
in your blood there's iron everywhere
like i'll pull that out and just like re
reorganize the world to make a bunch of
these and it seems
almost whimsical but i guess one of the
challenges i've always had is like
is conveying the fact that this actually
applies to far more mundane things like
you know i think as you say yukowski is
focused on this question a lot i think
one example i saw him write about was um
you know we'll know that we have a safe
ai system when we can
get an arbitrarily intelligent ai to
just place a strawberry on a table
like you know something just like that
simple how do you define
strawberry and how do you find table
like if you have
a genie in a bottle that's arbitrarily
powerful like you know
it slight differences in meaning can
translate into
massive differences in in action um
is there like i guess you're not an ai
safety researcher per se but what's your
level of
optimism regarding whether we're able to
meaningfully
move the needle on this this category of
risk in uh in the next say 20 years
before before 20 40
20 47. um
yeah mostly i don't know
uh there are the philosophy at open
philanthropy is that if we have a small
chance of making a big difference
then we should try it unless you know
the chance is so small that it's
you know at some level it just seems
pointless uh
and so that's that's the philosophy
we're going with
uh i i forget what the internal
calculations are but you know i think
it's something like we think we have
less than a 10 chance of making a
difference but
if it's if it's to tempest if it's a you
know one percent or five percent chance
of
shaping the next economic revolution
that's a great
use of philanthropy philanthropic money
um
so mostly i have dodged your question uh
um
yeah i i i
no one really knows i'm i'm optimistic
about work
that's being done on on adversarial
examples
because what i think it is helping us
understand is
um and also on work that tries to
actually go inside the black boxes of
these deep learning systems and figure
out what
what algorithms they're settling on
because adversarial examples are sort of
in some ways they're the canary in the
coal mine they're showing how small
seemingly small changes in input can
lead to very large changes in output
suddenly a picture that looks like an
apple does
is classified as a panda bear right
and the fact that we're on top of that
now or working on it now is i think a
good sign and we are gaining insight
into how these systems actually
think yeah and then maybe they'll
if we can demystify what's going on
that'll make it more easy to manage
yeah that makes sense it's certainly
like a whole family of of strategies
look like clarity and and i think chris
ola
formally at opening eye now i think he's
doing his own thing but um yeah
that's definitely a research program i
i'm excited about too um
now going back to this idea of timelines
like were you surprised
by first off keeping in mind all the
regular caveats apply 2047 is
you know the result of modeling so you
know who knows
but let's just say like the order of
magnitude like that 2047
isn't more than it's not a lifetime away
i mean it's on the horizon
did that surprise you and and if it did
like how did that affect your thinking
about
almost like i want to say life
priorities in general i mean there's so
many things that that number implies but
um
yeah i just want to get your thoughts on
that yeah
um it has not affected my
thinking on life priorities i'm not
expecting a singularity in whatever it
is 26 years
um
i'll say two things one is i think it
did help me appreciate
that the human system is unstable
that we're people maybe a lot of people
listening to this or watching this right
now you and i
we've experienced relative stability in
our lives things economically socially
are pretty similar to what they were a
few decades ago even
arguably a hundred years ago uh and
you know i ran different versions of
this model in some cases there was a
you know explosive takeoff in other
words some other cases there was
a takeoff and then a crash and it's hard
to make the system
uh under the rules i had given it
stabilize
so i thought that that was sort of an
important insight i'm not sure where it
leads practically
i will say more generally that this is
part of a larger effort on my own part
just to understand things i think you
understand at least as well as i do
about
the state of ai how it works what where
is it where it's going
and i think through that process of
understanding just understand
what is deep learning for example and
learning about the recent progress
that my sense about i i now see
uh human level ai however we want to
define that precisely
as more plausible than i did a few years
ago my mind has been shifted towards
plausibility you know when i came of age
it was laughable that a computer could
be a grandmaster in chess
right yeah and now it's now the reverse
would be laughable
yeah so i've had to adjust my own uh
priors and so i
i do think it's not uh i don't think
it's as crazy as i once did
and you mentioned the crashes in the
model that that is something that i
i noted with uh with less than glee as
well so
what what happened there i mean do you
have any um
is there any explanation you can think
of in terms of the
like the mechanics of what might lead to
those sorts of events uh as the curve
goes up
up and then all of a sudden yeah yeah
the the idea here also can be found
work such as the limits of growth and
limits to growth in the 1970s
uh in in the variants where there was a
crash i introduced another factor of
production
along with labor and capital and
technology i added natural resources
and i plugged in the assumption that the
more
economic activity there is uh the more
we deplete those resources
so if there's an explosion towards
infinity in economic resources then
within a microsecond i'm sorry it was
explosion and economic output
within a few microseconds you've
depleted all the natural resources
and under the assumptions of the model
the economy then has to die
you can't subsist without any resources
at all so it crashes
so that suffices to at least show that
you can have
a simple model that produces
accelerating growth throughout history
and yet does not predict positive
infinity going forward
or what that's worth makes me think of
uh naseem teleps um
that the happiness of the turkey over
time all of a sudden
um but what i will say is you know
caveats to that um
as a prediction built into the the
simple mathematical structure there is
the assumption that
people don't change anything about their
behavior
as the world changes yeah just keep
using resources at the same rate
relative to their economic activity
they don't adjust circumstances which i
think is pretty unrealistic
of course that doesn't mean we're
incapable of overshooting and crashing
just it's clearly plausible too and yeah
yeah interesting so so the in in your
view then
like if your your gut obviously nobody
knows and everybody's
you know poking at different different
parts of the elephant here but
um do you do you consider like the
future of humanity as being
like an either an unbounded infinity or
a crash to zero basically it's like one
of those two and there more or less
isn't really a middle ground
would that be uh your perspective um
no i know it's what comes across in that
blog post because that's where the
particular
simple models i was um working with
i i i would say i i would put
more expectation on you know a muddle in
the middle i think we're going to do a
lot of harm as humans
i think we are adjusting our behavior a
lot
maybe not in time to avoid substantial
harm but in time
to uh forced all the worst outcomes
um interesting okay yeah i'm i i find
myself
like when i look at those curves even if
i assume that the curve itself
is a curve of goodness and that that's
its own problem right because like
we look at these numbers we're looking
at gross world product or or
gdp or whatever the metric is um
and it's just like there's there's an
unease that i have even as i see that
number go to infinity
that has to do with i think my own
primal need for
for constancy and consistency and things
not changing too much
and it's like this is going to sound
really weird but i find myself
in a way almost rooting for the crash to
zero outcome
just because at least it's bounded at
least i can conceive of
of a universe where humans don't exist
whereas one where
everything is just like an agi that's
gone crazy and uh
or whatever whatever form that would
take i don't i don't know i mean at an
emotional level
there seems to be that resonance to it
yeah i know i i share that i
understand what you mean maybe i should
explain that one thing that was going on
in my work is
that i was incorporating into these
mathematical models
something called the stochastic calculus
which was new to me and is often used in
modeling stock markets
it's a way of bringing randomness
uh into calculus so
you know instead of having beautiful
curves like you learn about in your
calculus class
you have random wiggles with overall
patterns and i was
that to me just seemed like a very good
model good way to approach modeling
human economic history because there has
been an overall pattern but lots of ups
and downs
the black death and other other
pandemics actually in world wars and so
on
and that turned out to be such a
complicated thing to try to bring
to to to build a stochastic calculus
model
that had the overall acceleration that i
was looking for
that i had to keep everything else
pretty simple maybe maybe people who are
watching have that experience you're
building something complicated and it's
just an achievement to get the new idea
to work
that you don't want to introduce any
other complications until you've got
that piece working right
and that's kind of where i was and to
that ex that's part of why in
some of the the in other ways the blog
posts and the analysis is very
simplistic
it only predicts infinite takeoff or
crashing to zero
because that was a that was the level of
sophistication i could achieve while
still bringing in the stochastic
calculus
yes it's still interesting that it was
so consistent in its its output like i
remember looking at
you know when you're introducing your
use of veto calculus stochastic calculus
um and then you showed all those curves
they they did all have like a strikingly
consistent shape and and i think there
was one criterion you looked at too
which was like
you know by the time you reach a certain
level of
economic activity the probability of
this curve going all the way to infinity
starts to go up like really fast like it
seems like there's a
baseline level of economic activity that
we've
since long since crossed beyond which
you're like yeah now
now you're really into singularity
territory or whatever um
can you can you speak to that cut off a
little bit on what you may have learned
about it
yeah well the model i've developed has a
notion of depreciation
which is standard in economics this is
the notion that things wear out
people die so
you have to uh count you know
two dynamics and tension one thing is
one is that stuff wears out
at an exponential rate an exponential
decay
the other is that as the economy the
system grows
its capacity for innovation picks up
which then can accelerate growth and
what that means
is there's a rough cut off and if
humanity starts below it then it's
liable to just go extinct
you know as as most species do right
there never would have been human
history
and if it's just above it in a
stochastic model it can either escape
that threshold and take off or it can
wobble around it for a long time or
maybe go below it eventually and crash
all possibilities are possible and so
that sort of gives you a way of
representing the fact that for
millennium millennia there wasn't that
much increase in the scale of the human
system
until a takeoff occurred and then once
it did i guess
do you have a sense of like when based
on your models like
around even like what year we crossed
that threshold where
you know it now becomes almost
inevitable that that we get to 2020 with
computers and whatever else
um uh i think
well in in the graphs that i focus on in
the blog post i start
the world world the simulation of world
history at 10 000 bc
so 12 000 years ago and as i recall
it was something like 95 at 100 paths
that started where we were in 10 000 bc
with a very small economy
eventually do take off wow then the
model the best fit of the model
i mean it's sort of a post-facto thing
yeah you fit the model the data then the
model says the data
which you should get um it says that
there's a 90 according to the model
there was a 95
chance of eventual takeoff now in some
of the runs that take off
could occur 5000 years later yeah so
uh it's like they just happened to
invent the wheel 5000 years later and
everything kind of
well rolls on from there but it does
seem to imply that there's something
about our neural hardware
that was already present at that stage
like again assuming the model is correct
and i should yeah i i should reinforce
your your warning there this is all
it's it's all modeling and low
dimensional stuff but it's kind of
interesting that it has you think a
little bit about like
you know were all the ingredients in
place such that
you know the moment we we deviated from
the the line that led to chimpanzees
evolutionarily like
maybe even at that moment things were
baked in we were
always going to end up with the iphone
and it was just a matter of time like
it's sort of it's fascinating from that
perspective in terms of what it says
about us
physically as as computation i think you
can make that case
more convincingly if you go to the
development of language
which i mean i'm not an expert on this
but from what i read it looks like that
has been dated to 40 or 50 000 years ago
which is remarkably recent you know we
split from chimpanzees
what like five million years ago i can't
remember now
uh and so hominids recognizably human
species have been on this planet for a
couple million years
and only thirty or 40 000 years ago was
this thing called language
development and yeah it does kind of
seem like from that point on
you know there was a certain
inevitability about it i guess there's
also like a
zooming out question which says you know
we can talk about
what in the in the fraction of worlds
where humans arise
in how many of those worlds do you get
some kind of singularity some kind of
take off in the long run
but if you zoom out even further i mean
you could i could imagine asking like
like is that curve really the story of
humanity
or is it the story of apes
is it the story of mammals is it the
story of life itself
or even more fundamentally like does it
tell us a story about
whatever optimization process the
universe itself
is running like with atoms and molecules
and protons and whatever else
kind of mixing them together and then
like in i mean
this is a obviously the totally
unanswerable like fermi style
question but like you know in what
fraction of universe is do you naturally
just get a singularity is that
some weird feature and i mean
conspiratorially i might sound
conspiratorial here but like is that a
feature and not a bug even of this of
this universe like
is it designed to lead to that end or
whatever else um
it just it seems to invite so much
interesting speculation from that
from such a simple model too which is
what i find so exciting about it
yeah no it invites a lot it's true i i
tend to think i think that that's i
think that's
again well put uh i tend to think that
life is baked in that that's
you know it started so quickly on earth
in our his in earth's history
um there is a vein of scholarship that
says that's argues that actually
intelligent life on the other hand is
not so clear
that's if you know that maybe we are
really alone in the universe in that
respect
uh and the point the the data point for
that
is that uh the earth is expected to
become uninhabitable in about 800
million years
so intelligent life emerged on earth
uh what wherever it is like 80 of the
way through its habitable
lifespan and so it's just as we've been
talking about accidents of history and
the invention of the wheel
very easy to imagine for example the
meteor hadn't struck earth 65 million
years ago
that intelligent life would never have
evolved on earth
right that was an improbable event
and one can then contin of course
generalize the entire universe if one is
brave
yeah there's there's a lot of that that
does seem to have to happen yeah
it's really interesting it and humbling
too in its own way partly because it
always
seems to reveal our total ignorance
about all these priors there's so much
like
the numerator is one and we have no idea
what the denominator is type
type issues that keep popping up here um
now switching gears maybe to something a
little bit more day-to-day kind of
pragmatic because i know there are other
initiatives that um that you've been
working on that open phil's been working
on
and some of your your really exciting
work in this area has been like
looking at some of these studies like we
talked about the outside of the
conversation
from a skeptical lens and um
i i want to start with micro credit
because oh okay
my so my sister-in-law a random story
but she's from
bangladesh her her dad worked with
muhammad
muhammad yunus and who was the inventor
of or the
big player anyway in social credit and
um and so i ended up looking into this
and i was like oh this is fascinating
but but then but then i read your work
and i found i found it
it's so interesting to look at all these
different angles so would you mind
starting off by maybe explaining like
what um what micro credit is and then
what some of your analysis has shown
what some of the considerations are that
you're you're
batting around okay so
uh yeah i wrote a book called due
diligence um
which i'm rather proud of i think people
might like it
which is a an in-depth investigation of
microfinance which is giving financial
services to poor people
mostly in poor countries and ask just
asking the question does it
does it live up to its reputation for
being you know a great way to fight
poverty
um the modern forms of microcredit
well there's micro credit which is a
type of micro finance but it really
started
mostly with credit and there are reasons
for that
uh you know business and business
reasons for that
modern micro credits sort of emerged in
the 1970s through a few different
threads but a very important one
was in bangladesh with the work of the
man he just mentioned muhammad yunus who
went on to win a nobel prize for that
work
and he built something called the
grameen bank which scaled up to
uh to the point where it was making
regular loans you know
once or once or twice a year to millions
of people
mostly poor women in bangladesh
and this generated a huge amount of
excitement because along with the
numbers about how fast it was growing
there were stories of women who
got a loan bought a cow you know sold
the milk
gained some economic from independence
from their you know abusive husbands
this kind
of thing and so it drew a lot of uh
philanthropic interests although
muhammad yiddis himself wanted very
little support he wanted to be an
independent business he wanted his own
independence
but it became a global movement and
generated a lot of excitement
and i think to some extent is
responsible for the more general
interest now in what is called sometimes
called social business
different our social investing notion
that you can support businesses in
developing countries that are both
they're working like businesses but
helping people who need help
uh and so there did develop a bit of a
mythology some people over promoted it
and went beyond what the evidence really
showed about how much
how well it worked and you know so there
was this idea that
to use a cliche that microcredit was a
silver bullet against poverty
and um the surge just to make sure i
checked my mental model here as i recall
at least
when i heard it pitched to me the idea
was so we we tend to target yeah these
sort of more impoverished communities
women in particular because they tend to
think more in terms of community
interests
rather than men who maybe on average
will tend to be a little bit more
more risk-taking with their behavior
engage in in less
like community oriented goals is that uh
is that your fair cap
that's a theory yes that's a story that
was told about it whether it's actually
true is another question right yeah
yes you're absolutely right in
reflecting the the you know the kinds of
thinking around it
okay um yes in bangladesh in a lot of
other places
micro credit was done mostly in groups
so
uh five women would form a group and
they would all borrow the same amount at
the same time
and then they had weekly meetings where
they had to make their regular payments
which there were a lot of business
reasons for doing that it wasn't purely
it was described as cultivating
solidarity among women
using the mutual support but it was also
a way of improving efficiency since you
only had to have one meeting if you're a
bank officer rather than five
and there's role public shaming if you
don't show up with your loan payment
then the other women can't you won't be
released from the meeting
so then you become embarrassed and that
improves the repayment rate um so the
reality and the appearance were often
different things um at the same time so
i've already said some you know hinted
that
there was some mythology and it was its
benefits were exaggerated and i do think
that's true
at the same time i do think it's
important to understand
that if you are a poor person in a poor
country it doesn't just mean that
your income is low it's not like you
have an annual salary of
you know a thousand dollars a year that
you can plan on it means your income
is volatile and unpredictable right
and it also means you have less
insurance against emergencies like you
know if your husband breaks his leg
and loses he can't earn any money
um and so
uh poor people find all sorts of tricks
and techniques and informal ways of
managing their money
to navigate these very difficult waters
they may borrow with each other they may
give their savings to somebody they
trust just so they're out of their hands
it's out of the money's out of the house
and the husband can't spend it
um they may go to money lenders on the
corner
they will do all sorts of things to
manage and if a
large institution that um is not going
to disappear overnight
comes in and treats them like rick you
know treats them respectfully and offers
them a better quality service than they
can get from the corner money lender
maybe that's actually a good thing may
not change their lives
but i don't think it would be a glib to
just dismiss it as useless
one has to approach with humility the
decisions that poor people people
themselves are making
to manage their difficult lives so i
came to a rather balanced view
in the book that i think the project of
trying to improve
and institutionalize the financial
services that are available to poor
people
is a good thing like any attempt
to bring deliver financial services it
has risks especially
when it comes to credit yeah over over
enthusiastically
pre uh you know dispersed credit you can
easily get in trouble and that has
happened
in the micro credit world just as it
happened with mortgages in the united
states some years ago
so the more that the movement can move
away from credit and overly enthusiastic
delivery of credit towards micro savings
and micro insurance
the better off we'll be and i think
actually it may be an area where it's
better to put in less money as
a philanthropist because when lots of
eager investors want to put money into
it that pushes the system towards credit
yeah because and that's where it's
actually most dangerous
that's real okay that's really
interesting um because there's so many
analogs to
even our current market conditions like
you know when credit is really cheap
people
start throwing their money at riskier
and riskier things because those are the
only things that can generate any kind
of return and you skew the entire
financial apparatus
in a directions in some cases orthogonal
to where it should be
um what was in your analysis of the
upside i'm curious like
what was the um uh what were some of the
limitations on the upside what were some
of the areas where like the narrative
starts to break down a little bit well
one part of my project was to look at
the existing
economics research on the impact of
microcredit and this sort of brings us
back to data science
when i started the project there had
been no randomized
studies of the impact of micro credit
which was sort of you know it's kind of
the gold standard it's what we use to
decide
whether a drug is safe and effective we
randomly give it to some people and not
others
and that had been not been done with
microcredit or most other
things that are done to try to help poor
people in developing countries
but there were some non-randomized
studies where you just go and look at
people who had microcredit and those who
didn't and
in clever ways try to you know look for
instruments
if people know what those are try to
find natural experiments in the data and
argue that you have made a comparison
between
those who got microcredit and those who
didn't that it actually gives insight
into the impacts
and the leading study until the
randomized studies appeared
was something that muhammad yunus you
mentioned earlier had
cited like in scientific american it
showed that microcredit
did reduce poverty in bangladesh
especially
when it was given to women but the math
was really complicated
and i ended up spending a lot of time
writing my own code
to implement the methods uh based on
math maximum likelihood
and then working with jonathan mordock
rerun the original study and
eventually finding all sorts of problems
and concluding that its original
findings
was not accurate and probably a better
estimate of the impact was zero
wow and then the randomized studies came
along and mostly said the same thing
that if you look at the overall poverty
level it doesn't really seem to change
but there is there are bits of evidence
that suggests that people were able to
manage their financial lives you know
going hungry less often for example
that they were able to stabilize things
with these things that are difficult to
measure available to them
yeah interesting yeah i mean there's
some
there's something fascinating about that
category of problem and it's
it's one that you've explored in so many
different ways you have a whole bunch of
different projects that you've done
digging into kind of established maybe
call them like consensus ideas
and looking at whether they hold up to
scrutiny
and in many cases you know you mentioned
complexity it seems like that's a really
big
part of the part of the issue here where
anytime you have a problem that's
sufficiently complex
people if anybody's listening who does
actual like model building for data
science like
you know how much you can move the
needle with hyper parameter
optimizations how much you can overfit
like it really seems like this is one
issue where people end up shouting
different models at each other
rather than actually looking at data and
or looking at uncertainties between
models
um how do you like how do you develop
confidence in
in in that context and and moreover
actually if you can't develop confidence
like how do you make a call in in the
context of that level of uncertainty
right this is something that i think
economics has struggled with a lot and
maybe made more progress on than other
fields
in a couple ways although there's still
a lot to do
um you know one answer is that
uh then this may not really translate to
bread and butter data sciences that
one of the appeals of a randomization
randomized experiments is
not just that they're obviously
compelling but that they're
mathematically
simple you know you've got a control and
a treatment group you look at the mean
and the one and the meaning the other
and you
do some basic statistics of course like
everything you can start to complicate
it and
they do get complicated i don't know if
that up translates to data science
in general another important step that
has been taken it's a gradual
development is making research more
transparent
so i say that when i run the results
this way i get this
well there's a growing expectation that
i put my code and data online
right and that that allows people to
challenge me it may also change my
behavior and allow me to prevent me from
just tuning the hyper parameters too
much because i know
someone will figure out that you know
that was the key they're getting the
results that maybe
got me published so i think we need to
see a lot more of that in other social
sciences there's been a real crisis
in psychology arguably brought about by
the lack of
of transparency and research and there
may be a place for that
more and more in in machine learning i'm
not sure
yeah it always seems so potentially
counterproductive just because that
those hyper parameters the degree of
freedom that opens up is such
that you now it's almost like everything
has the risk of becoming political to
some degree where people have their
pre-existing notions and they'll say
like
okay i mean i can argue for anything
from the data um you know so
so i will right another innovation i'm
sorry
is and this i think uh i mean i said
economics has made progress
probably medicine was ahead of economics
another innovation is pre-registration
um yeah post your your analysis plan on
on a third-party site right which gives
you credibility that that
you can't they will keep a complete
history sort of like a github you know
complete history of that analysis plan
and
you can change the plan you can do
analyses that you did not plan
but it's clear to everybody what was you
know prepared and what was uh what was
done on the fly ad hoc
yeah and there's this it seems like
there's a delicate dance between like
um between confirming things like
running studies
that we almost know the answer to
because like we can pin them down so
much
and then running studies where it's
almost like we're learning
more than we should not more than we
should but like we're exploring
a space that's too unexplored where
anything that we learn can be over fit
to like
there's just not enough to to chew on um
i'm just remembering
the other end of the spectrum i've
mentioned this actually um
once before on the podcast but when i
was doing graduate school in physics
we would run experiments whose results
we would
begin by knowing i mean this was the
default right the number of times that
you should expect a violation of like
schrodinger's equations or like general
relativity in a physics lab
i mean that that is basically zero and
one interesting consequence of this is
you actually have
knobs that you can move around on your
experimental setup
and you naturally have a temptation to
say look what is the
prior probability that i'm going to get
a result that doesn't accord with like
newton and einstein and so on it should
be zero so what you find yourself doing
is going oh man i'm not
i'm not confirming my priors okay i need
to tweak these knobs
and you basically overfit until you
confirm everything that you always
thought you knew
and in a weird way this establishes even
more trust in a foundation of beliefs
that
could be shaky um if people just dared
to accept
sort of out-of-band experiments i i
wonder if there's sort of a similar
self-perpetuating dynamic in the space
too where you know
i guess one way to ask this question be
how often do you find yourself
challenging a body of work not just an
individual paper but a whole bunch of
studies that seem to
be following a consistent narrative or
or even
doing running consistent or getting
consistent results
from from data that seems to be wrong
yeah it may be different in economics
because we don't really have any laws
that anybody believe
uh you know there's a lot of supply and
demand but it
you know even then it gets it so far
yeah i mean i should explain that a lot
of what
what my career has been and this again
not planned it's what i stumbled into is
um has been the kind of pattern i
described with the micro credit study
where i come across a literature that
speaks to a question
that i've decided to investigate like
does putting more people in prison
reduce crime
right or does giving micro credit to
poor people make them better off
and i feel like well if i'm going to be
an expert on these issues
then i need to really understand these
studies and
when there's controversy or complexity
in them i have often felt driven to
actually rerun the studies myself
obtain the data or reconstruct it run
the methods and so on
and i have no advanced degree you know
but people assume i'm an economist and
i'm not so i'm kind of an outsider to
the field
and maybe that's part of why i've gone
this way i just i'm just the world's
most annoying consumer of research
and so my pattern has often been to look
at a study and say that looks pretty
good but maybe i should just check or
run it this way
and about half the time i end up being
uh unconvinced by the original claims in
the study
i don't think it's because they were
just trying to you know conform some
well-known economic laws
tends to be more that if you get a
non-zero result that's more publishable
if you're a junior you're and you're
looking for tenure
you're going to be especially drawn to
finding those non zero results
there's rewards for mythological
sophistication
which can often backfire i found there's
a bunch of forces at work
and what do you think because this seems
to imply some pretty disturbing things
about
what we think of as expertise and
what we think of and who we think of as
experts i think maybe that's more to the
point
the world's so complicated right i mean
we need to outsource our
our dimensionality reduction of like
these incredibly complex problems to
people who are specialized but then
if our process for identifying
specialists is
peer review um that seems not to work
terribly well so
not that i would expect you to have any
solutions because this is like the
probably the biggest open problem in
human understanding the world but
from a sense-making standpoint like how
how do you see this playing out like
do you think we're naturally going to
evolve better systems or
or do you think i don't know what do you
think
no it's true you know we have experts on
all sorts of things in our society
engineering criminology and so on
all of almost all those experts function
inside human communities which we know
create all sorts of weird incentives
because
you worry about offending your peers and
getting tenure and getting published and
lots of other things and there's a
tendency to go along with conventional
wisdom
but there are also rewards for saying
something that nobody else is saying and
so i think
these communities of knowledge do
self-correct over time
not nearly fast enough i've just
described some changes that are
happening in
economics and i think are slowly
spreading to other areas
but there are some real problems and
i've wondered whether
there's just a limit to what academia
and publishing and academic publishing
can do
to be more specific i was struck you
know so i took this question you know uh
the style of analysis that i just
described to you i took it to the
question of incarceration and crime
um as a part of an act of critical
thinking about grant making we're doing
at open philanthropy
around criminal justice reform
and i was struck by the end you know i
was able to
rerun about eight of the core studies
and four of them just
i think i took apart i just think they
were wrong
um and one of the ones that was right it
was like so obviously right it wasn't
it wasn't a randomized experience but it
was so close so you just look at the
graphs it's like clearly right
end of discussion um
and so what i realized is that these
these pieces of research at least their
goal was to influence
influence policy making around millions
of lives
billions and billions of dollars of
spending and they had received peer
review
that you might value at a thousand
dollars yeah look at the value of the
time
that the economists who put into
reviewing it and from a societal point
of view
that cannot be optimal we're spending a
thousand dollars
on to review research that with billion
dollar implications
i don't think academic publishers can
fix that yeah and i'm not even sure how
much
academia can directly fix it you know if
if that means doing more thorough review
i'm not sure they have the money or the
incentive so it might require
governments and philanthropists to be
engaging more systematically
in in-depth review of research that is
relevant for policy
yeah that's really interesting in one
sentence there i think you brought into
sharp relief the whole thing i mean yeah
how how are we spending a thousand bucks
to uh to review a two billion dollar
idea or something yeah
wow well so much to chew on i really
appreciate it i know
we're actually a little bit over time
here so i i will i will
uh let go of the reins here but thank
you so much for uh for your time on this
david it's fascinating conversation
i will flag for anybody who's interested
the book is due diligence so if you want
to check that out we will include a link
to it in the podcast
sorry in the blog post that'll accompany
this podcast as well um
david thanks so much and do you have a
website that people can go to to follow
your work
oh i do have a site davidrodman.com um i
don't maintain it that actively but it's
sort of a base if you're trying to
figure out who i am
great okay we'll link to that as well
just so people have it yeah thanks i
really appreciate the chat this is a lot
of fun
it's been a real pleasure awesome so
i'll stop the recording there