Tech talk: Privacy in AI safety

everyone my name is mark world I'm and
with me is Christian and we're going to
talk to you today about previously AI
safety it's going to be a Tech Talk I'm
going to keep it heavy on intuition
examples as well brief agenda so
introduction to speakers we've kind of
covered that we're going to do cycle T
briefly and I saved in how we think
about it then Christian will talk about
private synthetic data I will talk about
creating private synthetic data our
differential privacy and then Christian
will talk through some real-world
examples of how we've deployed this in
the real world if you have questions
throughout the presentation please use a
question answering facility on zoom' and
we'll try and run through as many of
them as we can at the end so we're
aiming to present for around 40 to 45
minutes and then take as many question
answers at the end as we can so as I've
said I'm mark and representing with
Christina both data scientists faculty
and faculty exist to make any real we've
done this through many data science
projects with many companies across a
range of countries and sectors and you
can see there on the screen some of the
companies that we've worked with to
achieve this goal so I'm gonna have a
quick give a quick introduction to AI
safety now and this has been a focus of
our research and development team for
for several years now essentially
because we believe that to make it AI
real has to be safe it's a fundamental
prerequisite in our eyes when people
often think about risks associated with
AI if we kind of zoom out and think of
broader risks they can be pretty
unstructured and cover many things so
things like killer robots which may be
in the thing of science fiction or
significant and really along our time
horizons and so things are with us today
you such as deep fakes
faculty we kind of loosely placed these
risks into a grid here you can see on
the x-axis intention and then on the
y-axis like the autonomy level which
roughly
paralyzed with the time horizon this is
just a brief framework and how to
structure some of those risks and
thinking around it as a company when we
think about AI safety we break it into
four four pillars the first of these is
explainable and that means like roughly
your to see what you think it means
sight you know can we demystify the
black box or some extent and explain
outputs and decisions made by the model
fairness this is more complicated to D
to define and can vary with societal
norms or with jurisdiction a faculty but
research which can essentially make
models fair conformance any of those
definitions robots Nexus data scientists
is something that we think about in
terms of generalization but also can be
estimating uncertainty from the model
even down to things like am i robust to
adversarial attacks privacy which is a
focus of this talk again can be hard to
define we're going to do that later for
love it I'm per one definition but also
can just loosely be thought of as you
know can we extract sensitive
information from this data set or our
individuals inner parts of our training
data compromised in some manner so with
that brief overview after the way I'm
gonna hand over now is Christian has to
introduce you to private synthetic data
right um just to quickly say we now have
fifty seven participants in the call so
I think this could well be the pinnacle
of my career okay private synthetic data
next slide please
so why does data privacy matter so many
of you are where dad's in real world
data sets we quite often have the
situation where we have sensitive
information in there and a prime example
of this is for example in the health
care sector so we have loads of datasets
where we have patient data and that
sensitive information that shouldn't be
released and just to really be explicit
about this why I shouldn't be released
well if this sort of data falls into the
wrong hands it kind of malign
consequences
individuals in the data so for
healthcare data set the classic example
is if your insurance companies know that
you have a particular disease they might
have higher premiums for your insurance
so what this means in practice is that
we really have to make sure that access
to those data sets is strictly regulated
and that unfortunately has as a
by-product this it creates this barrier
for any deployment of AI so the question
is then how do we still make use of this
valuable data while also simultaneously
protecting the privacy of the
individuals inside the data next slide
please
right so just to start with a really
simple example I will show you why data
on an organization by itself does not
actually guarantee privacy so this is a
really important point that is sort of
like the starting point in terms of why
do you actually need some fanta data so
imagine we have this small data set as I
show it here so this could be thought of
as medical appointments and as you can
see we have appointment numbers we have
some names we have tons and we have a
like a GP practice as well so if you
wanted to anonymize this data set what
he could do is he could get rid of the
column that has the name field next
slide please
right so we do that we get rid of this
column and the question is does this
data set now protect the privacy of the
people inside it next slide please
I'm gonna argue that is not the case and
I'm gonna show you how this could not be
the case so imagine now in addition to
this first dataset we also had some
auxiliary data so this could be for
example data from transport so here I
mocked up an example where we have
a tube station datasets where we show
different people executing a tube
station that's nearby the practice and
as you can see in this case we find that
the name b dot c dots exits the tube
station at very similar times to their
appointment status appointment time so
this means if we were to have access to
this data set we could to some extent
reconstruct this sensitive field in the
original data set next slide please
so this means if you just remove a
single column from a data set it remains
liable to we identification so it's a
really important point that just
removing sensitive information from data
does not guarantee privacy another
example of thinking about this is
imagine each row in your data set is
like a fingerprint and even if I remove
parts of that fingerprint you still have
quite a lot of information in there so
you could in theory figure out who the
road belongs to cool next slide please
yeah and just to say this is not just a
contrived example that I dreamed of this
has actually happens and we have a few
real-world examples here that you can
look up the most famous one is probably
the Netflix price where people were sort
of given this open data set of Netflix
movie recommendations and then which was
supposed to be anonymized but then
researchers found they could actually we
identified as using another data set
cool next slide please
so now I'm going to talk about synthetic
data and widest offers help in this sort
of situation so let's define synthetic
data first imagine we have some real
data here on the left and this looks
like some census data where we have
names ages and occupations and now
imagine we have some sort of mechanism
that makes synthetic data next slide
please
so here we have some private synthetic
data and as you can see we have sort of
similar information there we have the
same columns we have similar fields in
there
and you go to the next slides what's
important here is that even though like
all the data all the rows in the data
are different
what's the prime synthetic data does it
still is sufficiently like the real data
in many aspects so it manages to capture
like key statistical properties for
example the mean age so if you were to
compute the mean age in this left hand
and right hand data sets you will find
to come out the same quick footnote here
when we did this in rehearsal I actually
managed to get those numbers wrong but
I'm pretty sure they're now correct so
if they're not you know let me know cool
next slide so privacy how does this
guarantee privacy so mark will formalize
this later on a bit better in terms of
mathematical terms but essentially what
happens here is that when we generated
the synthetic data we didn't really have
any dependence on a single role in a
data but instead it was all about sort
of extracting statistical properties
that sort of remain constant without
having a single row included or excluded
so based on that we make the synthetic
data so it's not dependent on like an
individual in the data it just captures
aggregates properties next slide please
yes and finally always we have to
remember we're synthetic data is if you
really want to know how useful is my
synthetic data you always have to think
of a specific task in mind so the
utility always needs to be assessed with
a specific task in this example we saw
that the mean age comes out the same
thing if I fudged it correctly but if
you were to find like the fraction of
data scientists in this example we find
that in the real data we have 2/3 but in
a synthetic data only 1/3 so we have
this synthetic data set would be great
for computing the mean age but not so
good for finding the fraction of data
scientists in the population and I think
that's all from me and now we had over
back to mark who will go through the
glorious mathematical details
Thank You Kristen so I'm going to walk
through now an introduction to
differential privacy which is a
framework which we're going to apply
here to give us a guarantees that we
require I should know that every time
I've given this presentation so far I
always seem to mess something up so I'll
try and catch myself if I do that today
so Kristen is already alluded to this
fact that traditional methods like
anonymization or aggregation and don't
protect privacy and we can kind of think
of this loosely as this fundamental law
of information recovery which is a
overly accurate answers to many
questions destroy privacy and
intuitively that makes sense to us we
always have some friends who when we ask
them a question and so precisely and we
can easily extract the information which
we require from them and other people
who either on purpose or not just
answering precisely two things and it
takes as many more queries to get the
information that we require if we get it
at all and some of these attacks can be
applied to machine learning models that
are trained on data we can actually work
out if we have access to that model to
query of our own API tools or if we have
access to the model and weights all
parameters itself we can actually find
out sensitive information from people
from the training data some of these
attacks are sophisticated and they do
require a significant effort um but
regardless if you're not using formal
framework you have no guarantees of
privacy and this is precisely what
differential privacy will give us it
will give us a mathematical framework to
think about privacy alongside precise
guarantees so we talked about previously
being a concern for models and not just
for data and we can loosely think of
this trade-off in terms of privacy and
utility on the y-axis we have privacy
here from bottom to top we have no
privacy to maximum privacy and on the
x-axis we have no utility of our data
over moved to the right we have maximum
utility so there is a trade-off between
how private we want to be
how much your to litter we want to
retain and if we think of starting in
the right bottom corner of this chart
which is where we have maximum data
utility and we've done no privacy
procedures and that's where we usually
usually are and if you move up the
privacy plane then we give up some data
utility obviously we would love to be a
situation one with maximum privacy and
maximum utility but typically there'll
be some trade-off and we're going to be
able to quantify that trade-off and find
an acceptable trade-off for our use case
and that will vary per our use case in
some realms it's obviously much more
important to keep data private than in
others so just to step back for a second
Before we jump into the technical
details and to think about how we should
think about privacy even we want to
respect the privacy of individuals in
our data I first thought might be that
we shouldn't be allowed to learn
specific things about people within our
data but that turns out to not be quite
right well we can illustrate this with
an example let's say we have a friend
Bob who smokes and we then find out by
the internet through some scientific
study that there's a link between
smoking and cancer
so two questions arise from this first
like you know has Bob been harmed by
this scientific study possible it if his
insurance company knows he smokes as
well and they see this study they made
but his premiums up which isn't good for
Bob but has he's previously being
compromised differential privacy will
say no with the rationale that the
impact on the smoker Bob is the same
independent of whether they were part of
the study or not it's a conclusions of
the study which have harmed him not his
presence in the data set or not for this
in mind it's like Fitz saying that we
can reach the same conclusions from any
analysis independence of whether we take
an individual replace him with another
random member of the population so R
allowed to learn facts about the world
we're just not allowed to learn
something about an individual that we
can't learn without them that's what
that top boy point is saying and that's
like the key strap line for differential
privacy and we're going to see later how
that's aligned with any machine learning
like preventing overfitting and it
actually and will help us generalize so
in this sense differential privacy
doesn't save you from harm it saves you
from any additional harm that can arise
from being a member of a dataset and to
do this we've already alluded to the
fact that we may have to be slightly
less accurate in our responses and we
can illustrate this by the randomizer I
randomization procedure and coin
flipping and this is something that
researchers have been doing for many
years so let's say we want to connect
collect accurate information about
whether people are performing certain
acts I'd like to say this act is
something like drink-driving that people
don't want to admit to and therefore
it's difficult to get reliable
statistics around it so we can do this
and also protect an individual's privacy
by like play in the following game we
can give them a coin that we know the
bias off and let's just assume for
argument's sake it's a fair coin then we
send them into the corner and we tell
them to apply steps one two and three
and we don't see the outcome of their
coin flips they just come back to us at
the end of this procedure and say yes or
no
whether they've committed act X so
precisely they for this con once if its
tail's they respond truthfully if it's
heads they flip them again and respond
yes if it says no if its tail's if you
have to sit and work out what that
procedure gives you you'll quickly
realize that we can actually work out
accurately the proportion of people who
have performed at text so that allows us
to get accurate statistics about the
population as a whole
equally given that any person who
flipped these coins and we didn't see
which quasi flips had a 25 percent
chance of getting two heads and
therefore answering yes regardless of
whether they've committed at X or not
you've given any person the ability to
just turn around say you know I got to
your heads didn't actually drink drive
so we're giving them a sense of privacy
through possible deniability the key
idea here is that randomization is
essential for any privacy guarantee so
with this in mind we're going to
introduce differential privacy
specifically we're going to introduce
epsilon differential privacy there is a
more relaxed version of differential
privacy that exists called epsilon Delta
differential privacy Buddhists it's
easier to introduce epsilon differential
previously the ideas are identical so
I'm going to introduce the terms on this
slide first then we're going to walk
through an example
straight after so data sets D and D
prime are essentially the same apart
from one row in D prime has been either
added or deleted verse data set D
epsilon is our privacy loss or post e
budget depending how you think of it
this mechanism M from machine learning
we can think of this as a machine
learning model that's been trained with
the addition of some differentially
private algorithm so we saw some
randomization procedure applied to it
s is just an event or an outcome from
our model so what the left-hand side of
this equation is saying is that the
probability that I see an outcome s from
my model train and dataset D is less
than or equal to C that same output from
my model trained on a different data set
D Prime crucially only different by one
row with this factor e to the epsilon
and if epsilon is zero then the left
hand side is equal to the right hand
side and the output of the model doesn't
change depending on whether the training
data set D or D Prime and this the
mechanism M as introduced randomization
okay so like holding that thought in
mind and we're going to look at an
example now
so let's say we've taken our model and
we trained it on dataset do you and this
is a coral world and the true answer to
a particular query to our model is 100
but we know from previous discussions
that we can't answer 100 if we give
precise answers out that are true
then we're going to reach privacy
straightaway and the extensive
information so we're going to have to
add some random noise to this this
answer of 100 and that's what the coral
distribution is showing you it defines a
distribution of possible answers around
the true answer of 100 the great
distribution represents the world in
which our model was trained on a data
set D prime where one individual was
removed from the dataset so now let's
say the true answer is 99 but again we
need to release announcer from my
dataset or our model return announcer
and from a distribution around that true
out set and that's what the grade
distribution is depicting there if we
are on the x-axis and we pick a point
when we return announcer for our model
it will be deterministic after the
random training procedure has been
applied let's say we pick a point
somewhere on the right of this
distribution and we have 250 as the
output time model the point here is that
150 is a response the probability of
that event occurring is basically the
same in the coral world is in the green
world so the chance that an event occurs
with individual with an individual's
data and without their data is
essentially the same and when I was
saying loosely essentially the same here
that's precisely what the ETA epsilon
term was given us on the previous slide
so we can quantify this this banding
term exactly and I also made the point
that epsilon equals 0 would make the
left hand side on the right hand side
equal to each other here and we may
initially think that's desirable but
that's not actually the case
if the output of our model doesn't
change as we remove individual from our
data
and we've essentially learned nothing
from my data because we could keep
removing individuals one by one and so
there were no individuals left in our
data and our model output hadn't changed
and that would imply we learned nothing
from our dataset so the model output
will change as we remove rows from our
data differential probe is you just
bounce precisely by how much this is
allowed to happen and so we talked a
little bit here about being imprecise in
our answers so with the coral
distribution where the true answer 100
and we may have released any answer from
a distribution around that and that begs
a natural question of you know how fat
should we make these curves or how
inaccurate should we be when we provide
a response and that roughly depends on
two things one it depends on how
sensitive the models output is to a
given individual in the dataset let's
call this term Delta and also like how
private we want to be like this is
epsilon epsilon small that's highly
private so let's again say epsilon 0.1
then the heuristic is that standard
deviation of these curves should scale
in proportion to Delta over Epsilon and
we can understand this as follows if we
want to be highly private ie we wants a
very small epsilon then Delta over
epsilon is epsilon tends to 0 will blow
up as a term and the standard deviation
is curve will be huge or group will be
providing very inaccurate answers to
responses but will also be preserving
privacy if I'm highly sensitive to a
given individual the Delta is large and
there's a numerator term then the
standard deviation also increases if
epsilon were very large itself then that
would correspond to the peak
distribution in the middle where were
you be providing low privacy and being
very precise in our responses
so that's like a whistle-stop overview
of differential privacy as a definition
so I'm going to talk now about some of
the formal guarantees that the
differential privacy gives us we have
the ability to quantify the privacy
guarantee exactly for both individuals
and groups if we are Absalon
differentially private with respect to
an individual then we are k times
epsilon differentially private with
respect to a group of size k it for a
bus to all future attacks this is called
being immune to post processing and it
actually applies regardless of the
compute power you have differential
privacy is a definition but it's
programmable so different algorithms can
implement differential privacy in
different ways but crucially is
composable which means we can take the
outputs of datasets or models that are
differentially private combine them and
be guaranteed that the output yourself
will be differentially private this is
going to be crucial we think about the
implementation that's commonly used
called differentially private stochastic
gradient descent which essentially
ensures that with respect to any
mini-batch in the data
and we are differentially private and
therefore we're going to be
differentially private with respect to
the dataset as a whole this is called
composability in fact we can now
quantify privacy and do it exactly
allows policymakers and wider
decision-makers your own data to
quantify is actually the trade-off
between privacy and utility and as I've
already alluded to like privacy and
generalization are aligned goals not
memorizing or learning specific things
from an individual that don't apply to
other individuals in the dataset will
stop you overfitting differential
privacy is also called the gold standard
it's actually going to be used for the
2020 u.s. census which is highly
exciting if you want to read more about
differential privacy there's a free
online pdf by Cynthia Dworkin our own
Roth called the algorithmic foundations
of differential privacy if that's quite
formal and definitely assumes like
mathematical and computer science and
backgrounds but if you want to get a
gentler introduction then given that
differential privacy is actually being
used now for the US Census Cynthia's
walk has provided some YouTube talks
online which are very accessible
motivating the intuition and ideas
behind differential privacy
so that's differential privacy and in a
nutshell let's talk about quickly about
how we can combine this with generative
modeling I don't have time to introduce
generative modern I'm going to in one
slide in about 30 seconds
explain what ovae does so v8 eats data
in has an encoder which is a neural
network and essentially lonesome
compressed representation which
essentially parameters as a probability
distribution and then from something and
buy something from this probability
distribution and passing it through a
decoder whose weights alone during the
training process we output
[Music]
synthetic data at the other side to
combine this with differential privacy
is through an algorithm called DPS GD
differentially private stochastic greens
in and we've kind of got a simple slide
that introduces this idea here so we
previously talked about this Delta term
has been the sensitivity of our models
output to a given individual what DP SGD
does and there's a reference paper there
at the bottom you can see is clips
gradients whilst we're training the
neural network to a known norm this
bounds our sensitivity to any
individuals within the data we know the
norm we fixed up norm we then add noise
the amount of noise that we add will
give us a certain improved a guarantee
we want to be more private
well add more noise less private last
noise and that's what kind of being
depicted in this chart here we're
ignoring here any stochasticity that
relates to mini matching and on the
right we're just showing that a gradient
steps were kind of noise in the
gradients and so we may wander
occasionally and in the wrong direction
but over time assuming the noise added
is imputed to the training process we
will still converge so as an overview
this is what faculties approach to
private data gives us we have real data
in we train a differentially private
variation autoencoder
and we output either a private synthetic
data set or differentially prime over V
a crucially we have epsilon and Delta s
surrounding this and so we can have a
formal proofs are guaranteed which is
quantified the trade-off between utility
and privacy um with the overview in mind
I'm going to hand over now back to
Christian to talk through some
real-world results right thank you
mark so just before I guess that with
the next section just one to quickly
call-outs
there is a Q&A button that you can press
in zoom so if you have questions that
arise and
as we go for your presentation please
put them in there and then we consider
at the end go through that list and
answer them one by one
cool onwards and upwards so now that we
have a good understanding of
differential privacy let's go to sort of
the meat of the talk where we have the
nice results of how we actually use this
in practice next slide please
right so I'm gonna show you one case
study this was a project that faculty
did with M rat so M rad is the East
Midlands radiology consortium and their
problem is that so they have breast
cancer screening appointments for women
and the problem is that quite often
people don't actually show up so about
30 percent of people don't show up to
their appointments so this is obviously
not great from an operational
perspective because it means you have
some inefficiencies in terms of you know
staffing and not making sure that
resources are used well so the solution
to this sort of problem would be to have
a machine learning algorithm that for
example predicts the likelihood of a
no-show appointment and once you have
that it can then take an intervention
for people where you have low attendance
likelihood so this could be sending them
a text message or actually giving them a
call to make sure they actually show up
so that's the idea in terms of the
business problem however this data
obviously contains highly sensitive
information
and as such should never really leave
the sort of secure and rad environment
so the question is how do we approach
this in this case next slide please
right and our solution to this was to
have a machine learning model that is
trained on the synthetic data and then
deployed on the real data so let's go
for this diagram step-by-step so here we
have sort of two sections on the Left we
have the ambrette environment so these
are imagine a computer sitting somewhere
in in mrad and then on the right we have
two faculty environment imagine a
faculty computer
and in between we have this sort of line
which I call Primus emotes and this sort
of means that any sensitive data it
should never really cross this line so
we really have a good separation between
the am read and the faculty environment
okay so how does this thing work
so if we start at the top left you can
see we have some real historic
appointment data so this means we have
appointment data that sort of shows us
previously if people showed up or not so
some features and an outcome what we
then do is we take our sort of synthetic
data generator of private synthetic data
generator and trained us on this real
historic data so this happens all inside
the MRAP environment once we have
trained this model we can use it to
generates a bunch of private synthetic
data and that's by definition is then
safe to leave the ailment environment so
we go across the mode and we have
private synthetic appointment data now
in faculty environment and that means we
can just use this data and to our behest
and just train a machine learning model
to try to predict the appointment status
entirely on the synthetic data once we
reasonably happy with that we sort of
validated our model well enough we can
actually say ok this model seems to work
well let's try to actually get it back
into the unwrapped environment so this
means we crossed the privacy modes again
this time from the other side and we
just take our model and start deploying
it in the Emirate environment now that
this means we have a model in the mrad
environment and what we can do is we can
take the real date sir again so this
could be current appointment data that's
coming in right now and then based on
that you can feed that into the model
and get a likelihood of attending the
appointment out so this is sort of the
way we approach this problem and this
provides us a way to make sure that the
sensitive data never leaves the secure
environment it's probably worth pointing
out that other approaches do exist so
you could also imagine something like a
federated learning approach
but the reason this is quite nice is
that it's quite a low-cost way because
all you have to do if you have to make
sure that the synthetic data generation
happens on a synthetic side and then
once you make the synthetic data you can
do everything you want instead of your
own environment so in terms of
computational infrastructure it's quite
scalable in that sense next slide please
right and here we have some actual data
so what I'm gonna show you now is bunch
of univariate comparison between real
data and synthetic data that sort of
broadly assess the quality of the
private synthetic data and here you can
see a few histograms of numeric data so
on the y axis for all of these plots we
have to count so this is the number of
people in the data set and on the x axis
we have three different numeric features
we have a booking date/time so this is
like a timestamp feature a screening
appointment number so this would be like
appointment 1 2 3 it's a discreet
feature and then also we have time since
previous screening so this would be in
days one key thing to note here is that
this is not actually real data the real
data the real data in this case is a toy
data set that has similar properties to
the actual amber theta so this is more
on stratum showing if you have this type
of data you can do this sort of thing
and as you can see the histograms here
in red we always have the real data and
in grey the synthetic data and broadly
on all of these three plots we see that
they overlap quite well so we can see
like the peak falls in the same range
and we can even capture properties that
seem a little bit more difficult so for
example under plot on the right you see
we have this of almost bimodal
distribution where you have a peak at 0
and then another sort of Gaussian
feature at higher values so even that we
can capture with a method quite well
just to be absolutely clear about it
because some people have been getting
confused with these plots so the reds
and grey are the real and synthetic data
and whenever this sort of dark red color
is that means the two distributions
actually overlap so just to make that
clear
cool next slide please
right here we have some more features so
I showed you some numeric features now
we can also look at some other types of
features here on the Left we have
categorical data and on the right we
have binary data so for a categorical
data we have something like the practice
postcodes which has some like 80 values
in there and same story as with the
previously lived in America we can see
there's quite a high overlap between
real and synthetic data which gives us
confidence that synthetic data has good
quality then a similar conclusion can be
reached on the right with the binary
data where you see in this case it's
really spot-on in terms of the
distributions like you can not even see
any difference between the real
synthetic data in terms of these these
proportions so this is a univariate
assessment of the data which tells us
that on sort of a column by column level
the synthetic data looks quite good so
if you go to the next slide we can also
look at bivariate Association but in the
data and for this what we've done is to
compute the correlation coefficients
between features so here we have
Spearman correlations in the real data
and synthetic data and this sort of
tells you how much does one feed to
depend on another feature so if they're
highly associated if they're perfectly
correct it is one if they're perfectly
inversely correlated it's minus one if
they don't depend on each other it's all
in terms of Spearman correlation then it
would be zero so this gives you some
sort of indication of feature
interactions on the bivariate level and
as you can see again we're doing fairly
well in terms of reproducing what we see
in the real data so
so if these numbers always tell you the
correlation coefficient and the colors
indicates the value as well so just by
looking at the colors you can see that
looks quite similar and based on that we
have a good reproduction of these
correlation coefficients in the
synthetic data we can also look at
specific examples so for example if you
look at the booking date separation and
booking date time we have in the real
data we get minus 0.66 and in the
synthetic data we get minus 0.65 so it
seems super close and same thing happens
if we look at something that's less
strong in terms of the correlations so
if you look at correlation between at an
appointment status and screening
appointment number we have minus 0.04 in
the real data minus 0.08 in the
synthetic data so you see that's the
same trends are definitely captured in
the synthetic data even though it's not
perfect in terms of the actual numbers
so this means that the data obviously
has the same trends but it's not a
one-to-one mapping really because it is
ultimately a synthetic data sets that
will differ a little bit so these
capture to some extent univariate and
bivariate distribution of the data
what's important to note is that that's
obviously not everything that's going on
you could have much more complex
dependencies in your data and these are
generally quite hard to tease out with
these sort of simple statistical
exercises so as I was saying earlier the
best test of this data will of course
come from actually training a machine
learning model on it and then seeing how
it performs which we'll get to in a bit
next slide please
right so what I also wanted to show you
is we can actually compare directly rows
in the real and the synthetic data so
here you can see a table of one row in
the real data so on the vertical axis we
have all the features in the data set to
go down and compare to that we have the
row in the synthetic data and here the
ruinous effective data is actually the
closest match
and closest match here means that we
used the go distance to find from all
the rows in the synthetic data was the
closest one it's almost like a nearest
neighbor I cannot find the real Victor
so here what you can see is that for
quite a lot of features for example the
categorical features we actually get
exactly the same value for example
practice postcodes smart clinic we have
identical values in real and synthetic
data and that to some extent is by
product deaths you don't have that many
choices in terms of the values for
categorical features but if you look at
some numeric features like time since
previous screening or what else do we
have
previous screening dates you can see
that clearly differ times in previous
screen is like a hundred days apart or
every screening appointment number is 20
versus 11 in this effective data so you
can see that there are clear differences
and this is the closest matching of
synthetic data but despite that it's not
really the same so it's sort of
different in subtle ways but overall it
might still once embedded in a larger
set of data it will still look similar
on that perspective cool and I think we
can go to the next slide right so this
is the final slide on results and this
is sort of the key plots that we want
you to take home with and the headline
result is that higher primacy of
synthetic data leads to lower model
utility in deployment all stated the
other way around if you want a high
utility you can't have also super high
privacy so it speaks about there's
always this trade-off between the amount
of privacy and synthetic data okay so
what do you actually look at in this
plot so here on the y-axis I show you
the performance of the model and just to
remind you here this means the machine
learning model was trained on the
synthetic data and then tested on a set
of real holdouts data so this means we
helped back some real data at the
beginning and only used it at the very
and to test the machine learning model
that was trained on synthetic data and
utility here we had used this a measure
at the area under the ROC curve so 0.5
means the model is essentially randomly
guessing and then the best possible
utility of the model you can have is 0.8
in this case which is a model trained on
the real data directly and as you can
see in red line we have the different
synthetic data sets and the associate
model utility and as we have to very
high as privacy so epsilon around naught
point 1 or so we actually create
synthetic data that's practically
useless for this sort of classification
task but as we reduce the amount of
primacy and get to epsilon around 1 we
can see we actually gain a useful data
set and a useful model and we get the
cut an area under the ROC curve around
0.7 or so and you might say this doesn't
look amazing because in the real data I
get 0.8 but it really sort of depends on
your application point seven could be
great in terms of getting a model
deployed and having something that
actually works a lot better than what
the current system yes and also to
emphasize that here really the optimal
choice depends on your needs so maybe
for some use cases you don't have that
springe and constraints on privacy so we
can go to higher epsilon values and then
get better models but made for other
cases that's not possible
and a really high privacy means we have
to make some trade up in terms of the
model performance right and with that I
think we can go to the conclusion so
conclusion I've got three points here
first thing is making a r-real requires
safe ways to utilize data sets that
contain sensitive information so we
firmly believe that there's loads of
data out there that has sensitive
information but that is sort of locked
up in organizations all over the world
which should not be used as it is and
should not be used in anonymized form so
we think that technology here can really
make a difference in order to get to
somewhere we can unlock the value of the
data sets
and in particular we think that this
sort of approach of differential private
synthetic data can unlock the value of
these statuses without really
compromising privacy and we believe that
this sort of approach will is sort of
simple and generalizes to many domains
and datasets and by that regard
hopefully it will enable AI adoption in
many areas where privacy is really
paramount and I think this is the end of
the talk
yes it is so now the fun starts the Q&A
section I noticed we have quite a lot of
questions here that we can go through
how do we do this mark I can't see them
so you'll have to dish out them out I
think ok share just them so I'm gonna
message queue all the questions so we
can sort of make sure we answer
everything in a systematic way and then
bear with us
this will definitely worth it yeah also
feel free to answer more questions as we
go along if you have any more desire for
more information so I've got a couple
questions here one someone's asking if
probability of one dataset is zero or
the outcomes of a model training one
dataset is zero and on the other day sex
1 that was the big change that was bad
the point is precisely that bees the
epsilon term that bounds how much that's
allowed to happen so you won't be
allowed to happen and you could have for
a given epsilon such a large change
epsilon small you better understand how
much that ratio can change
there's also a question around like what
are the downsides of differential
previously compared to other purposely
approaches and definitely one of the
downsides is that it's it's quite a
formal and complicated scientific
definition from computer science and
developing algorithms that can scale
wasn't done until very recently so this
differential private stochastic gradient
descent is only has only been around in
the last few years and also trading
areas for datasets at scale and wasn't
possible until until recently
I'm also when you're adding noise I you
are harming the learning process to some
extent so differential privacy is not
around a small data set you do need
quite a large amount of data and also
like differential cruzi tends to work
well when you can process the whole
dataset in one go so if you've got data
coming in it's more complicated to
perform differential privacy then
because every time you're doing this you
have inside composer steps keep track of
your epsilon the Delta is to know your
current previously budget is and that
like accounting of the privacy budget
can get quite complicated so it
definitely is better to process a whole
dataset in one go home answers the
questions cool I can also answer a
question so there's one question on will
decides be given and I've been told yes
the slides will be given they will be
shared to all attendees and post on our
YouTube channel so that's good and
there's a question here about why was it
easier to train a VA in the secure
environment and then just training a
predictive model in the environment in
the first place and that's a fantastic
question and so you can train
differentially private classifiers
you could even just train a classifier
directly in their environment the real
benefit of training a differential
private and variational autoencoder is
you can then give copies of that data
set to your own data scientists so let's
say you're in an organization
where everyone internally can't access
the dataset and without some money no
clearance you could then take a
differentially private VA e train and
train on the sensitive data and either
give that model then to your data
scientists or to staff give the data to
staff who can then perform analysis on
the data set that's like a general
purpose use case versus like a
classifier which is just you know very
task specific yeah maybe just to add to
this um even if you were in a position
in your organization and you can do
everything yourself there's still you
know you might still want to consider
having a mechanism whereby you allow the
past other organization maybe even to
help you in terms of this is sort of a
data set that I really want to work with
I'm struggling with I want to see what
are the best possible methods out there
so it's just it really enables quite a
lot because it means that you know
access to this data set will be so much
easier than when itself is heavily
regulated cool I noticed we have another
question illusion actually yeah so the
question is is the ethical trade-off
between privacy and utility for
high-risk situations ageing medical
models were increasing privacy may lead
to worse predictions
how do people approach this trade-off in
practice um I'll have a start maybe you
want to jump in listed I mean this this
trade-off exists for the whole machine
learning right um
I don't think anyone here is advocating
that machine learning models should be
deployed into situations where their use
for critical decision-making and without
humans in the loop and so I proper model
validation is always crucial part of
that process and the fact that our model
is slightly worse and
because it's made private you know as
Christine is alluded to there may be
situations where it's no model giving
you no utility having a private model
which has some utility is actually
beneficial particularly that can be
combined with with other humans in the
process so I don't think there's an easy
answer to that and it's clearly going to
be something that's determined you know
both legal aspects and from the use case
for the companies and this is your
makers involved Kristen do you have
anything
I think yeah I think everything in your
answer one thing I would add is this
this is a really quite new stuff still
and not a lot of people I think are
actually even doing this or thinking
about doing this so even in something
like the US census there are discussions
in terms of what should the absalom be
so I think we probably still have to be
a little bit of the other work in terms
of to come up with guidelines and how in
which situation is what is applicable
and yeah I was able to pen from
situation to situation and but obviously
as we go along and as we sort of maybe
even have like these privacy attacks on
models where you can actually show that
whether model digs information or not
those sort of techniques will be helpful
in terms of really assessing for this
type of model I have this sort of danger
that's why I choose this absolute value
okay there's a question asking is any
classifier I trained on differentially
private synthetic data automatically
differentially private the answer to
this is yes this is a composability
point which is if you take any function
and perform it on differentially private
and data or outputs then you're
automatically differentiated private so
that's yes to that question
[Music]
we have more questions so oh maybe I'll
let you take the view you want and how I
was just gonna read it maybe then I'll
decide what I'll take it or not or if
the VA e in in the secure environment
why would you use that VA to generate
synthetic data examples versus just
passing the embeddings from the real
data over to the other environment for
modeling on latent data right do I
understand this well I'm not sure if if
the person is meaning will the latent
space respect privacy and I think the
answer that would be no well unless it
was differentially private and
differentially privately trained um I
mean I guess we could passivates in
space over and work with that but
ultimately we do want to build the
machine and I'm your model that's
probably interpret of all and to the
people so inference time they can pass
the real data in and the classifier
itself was trained on on the real you
know structure of the data with the
correct columns etc yeah I don't know
yeah so this is the in banks on the real
data over to the other environment yeah
I think the key point is if you have
embeddings on real data in a non private
way you know that's non private but if
you do it for synthetic data yes in
theory that's like since you saying you
pass your VA over to the real
environment which
in a way pauses a little bit more risk
because then you actually have access to
the data generating mechanisms own
by just giving you the synthetic data
that comes out in sort of limit the
amount of information you can squeeze
out of the model so it provides you
actually a higher guarantee of privacy
in this case yeah so on that point you
should be noted that the synthetic data
from a variation or two encoder is more
private than the view yourself because
it represents a phone and finding its
amount of samples from the VA II
obviously if you you can just keep
sampling and throughout that basically
there's a question around is there a
standard epsilon per industry or problem
type no is a short answer it will be
very interested to see what the US
census thinks is a sensible answer to
this question there is a theoretical
value of epsilon which is basically log
two and and if you don't go epsilon
below log two then in theory someone
could have still some chance of in the
future we are to extract something from
your and from your data should be
pointed out that from experiments we've
conducted you can take state-of-the-art
machine learning attacks from research
and conduct those on data sets and
models and eve of epsilon values that
are significantly higher so we're
talking in the hundreds here and you can
basically thoughts career machine
learning attacks and so
obviously not having an epsilon below
log2 means that in the future someone
can be smarter and design a better
attack or they could have more compute
power you're not going to be safe unless
your epsilon is below log - what very
high epsilon values at the moment and do
guard against state-of-the-art attacks
at present cool
I think that's answered so we have a
bunch more questions and we have one
minute left
so how do we approach this maybe pick
one more question to answer so the
answer to the question on is the latent
space guaranteed to be differentially
private even if the VA II was trained
with DPS GD and if the latent space is
output from the variational Ottoman code
it will also be differentially private
and I think it's important to know that
anyone who's asked a question and we
haven't had a chance to respond to and
we all be able to follow up with you
afterwards and email you responses to
your questions so I apologize we haven't
had a chance to to answer everyone I
don't know if Kristin you want to
squeeze any more in before we finish I
can see one question on how to make
synthetic data I think we're not gonna
be able to squeeze that in yeah I think
other than that we've covered everything
[Music]
yeah I mean there's a few
questions now but I don't even have time
to respond to them okay so let's leave
the remaining ones by email and yeah I
think that's everything from us then
okay thank you very much everyone
great thank you hope you enjoyed this
have a lovely evening everyone
[Music]


Artificial Intelligence Safety and Security - Roman V. Yampolskiy, PhD

[Music]
we'll get started
so I'm Romanian Polsky I'm a computer
scientist and engineer at the University
of Louisville my main area of research
is safety and security of intelligent
systems and how many of you actually
attended talk keynote by Professor Peter
a bill beautiful so you got the
prerequisites you can started telling
you about safety issues
superintelligence this is the follow-up
to that you don't have to know what he
said but it kind of builds on top of
that and I'll start from scratch so
there is no worries in case you find
this topic interesting you want to learn
more about my work you are invited to
follow me you can follow me on Twitter
and follow me on Facebook you cannot
follow me home for safety and security
reasons okay so what is artificial
intelligence how many of you seen this
slide before somewhere all right every
time we start with that all I'm trying
to say with that is based on definitions
we always head for this field starting
in the 50s we kind of got there the
problem with that is that the moment we
succeed we stop calling this technology
artificial intelligence things like
spell checker things like Google search
they all AI projects and very successful
ones but the moment we have it it just
becomes another piece of software we
stop referring to it as such so I'll
make some controversial statements this
is not a very controversial one under
those definitions we already have AI we
also know how to make bodies from
machines so we know how to make brains
we're pretty good at making robots
different manufacturing robots home
animations I like this
coffee guy he follows you at conferences
and gives you espresso that's a pretty
sweet machine so those two go together
the question immediately becomes what
happens next right so if we know how to
make useful to narrow
we at some point get them to be a
smartest humans smarter and perform as
well in general intelligence not just in
narrow tasks but really compete with
people in the areas of science and
engineering anything like that I would
argue that it's a definite possibility
and here's a few reasons so one we never
had so much funding for any type of
project from European Union White House
top corporations we're talking about
billions of dollars for reverse
engineering human brain for just
developing more advanced neural networks
no matter how you look at it really
unprecedented not just funding but also
human resources right the best people
smartest people working for those
companies all working in different
aspects of this technology we now have
conferences devoted specifically to
artificial general intelligence not just
AI there are books published and it
happen journals this was not the case
five years ago ten years ago so this is
very encouraging and it'd be surprising
if this never worked if they failed
after any amount of years now we're not
saying this is going to happen very
quickly and of course everyone
immediately wants to know well how soon
how long before super intelligence and I
don't really know and no one knows but
there are some people who do interesting
work on predicting what machines will be
able to do based on computational
resources available to simulate
different types of biological systems
whatever they're humans or animals those
projections showing that around 23 2014
5 depending on how you compute compute
you'll get enough resources to simulate
performance of a human brain and the
same person Ray Kurzweil he's a director
of engineering at Google
he was pretty successful at predicting
our milestones in the eyes development
for example he was very accurate with
predicting than chess will be lost too
much
Cheng's human world champion will Louis
and so on so the good news is even if
this is wrong if the dates are not
actually gonna match everything else I'm
gonna tell you still applies it's just
maybe we have more time to deal with
safety and security issues so what I
want to do is analyze what features what
properties those systems will have we
don't have them yet but there are
certain things we can kind of guess
about and based on existing trends and
just project them forward so just from
definition you're kind of suspecting
them to be super intelligent super smart
systems to get an idea of how that looks
like look at current AI systems and
narrow domains you all heard about
alphago and computers beating humans at
jeopardy and chess and all the sava
games the typical pattern is computers
are terrible they get slightly better
they quickly match human performance
they exceeded by so much no human is
competitive again we'll never here
we'll never have another human chess
champion for example so what happens
then this exact performance can be seen
in general domains cross domain
performance Science and Engineering
being the most interesting applications
of this technology so we can expect
exactly that we can expect that no
single human would be able to to compete
with this technology and a machine like
that or networks of such machines will
be as capable as any human in any domain
another property is complexity of such
systems so this is an image I grabbed
this is an instrumental panel for modern
airplane basically a GUI in front of all
the source code behind the outer pilot
all the sensors I don't know how many of
you have pilots licenses but I think
this is a bit confusing to keep track of
this is the easy interface of a pilot
the code behind it is much more complex
bigger and even experienced pilots don't
fully understand how it works in all
circumstances we've seen some reason
examples of airplanes unfortunately
having accidents exactly because of it
because we don't fully understand how to
interact with some of the more advanced
features about a pilot of this
technology and our property is speed we
all know that computers are fast that's
not what I have in mind here this is
what is known as ultra fast extreme
events machine can change your
environment so quickly that you don't
even recognize something happened so in
the case of stock market it can bring
down the market bring it back up
billion dollar shift in value you don't
even know this happens in nanoseconds
how do you monitor something like that
how do you respond to issues like that
of course we would never be silly enough
to give control over vital technology to
machines something essential for human
performance but actually we did a long
time ago and not even two very smart
machines we gave it just to dumb
software if you think about stock market
majority of it rates a fully autonomous
rate if you're thinking about nuclear
response if you're talking about power
great electrical grid all those
functions are now run by software and
there are too complex for humans to take
over in most cases we're seeing more and
more of this take place with autonomous
corporations with so many things being
connected to the internet businesses
becoming a stronger case for loss of
control to to software with my
background in cybersecurity I'm very
interested in computer viruses and
malware in general and there is a
general trend of because we have more
devices connected to the Internet the
viruses get a chance to in fact more
devices the damage is greater but all
this is kind of using standard exploits
what happens then we combine malware
with social engineering attack
capabilities we're starting to see
software capable of faking voice faking
conversely
patience but doing it on a scale so
before you had a spear phishing attack
you had to do your research look at
their social media you maybe hit one or
two targets now you can look at a
million maybe a billion targets at the
same time you get a tiny success rate
you know five percent click rate you're
still getting enough accounts to
penetrate any network many times over
and we know that social engineering
attacks are the worst
even professionals fall for them right
you get you get a phone call from your
boss telling you to do something you're
gonna do it or your wife depending on
but this is very interesting in that
we'll talk about deep fakes and more
recent capability service technology
essentially not being able to tell what
is real and what is fake is a huge
loophole for from a weakest point in any
security system a human user who's
funding a lot of this work right so
nowadays industry Facebook Google
provide lots of funding but historically
and still to a large extent military is
very interested in developing killer
robots autonomous drones so a lot of
funding goes to doing exactly that make
machines which are very good
exceptionally good at hunting down
humans and killing them there is also
good applications rescue robotics and
such but at the end of the day military
has certain applications they're working
on so that's a very direct example of
dangers from this technology where it's
not a result of accident but by design
so we can talk about applications of
this technology in war settings and if
it's beneficial to save human life so if
it makes going to war easier and more
likely but nonetheless the technology
itself is optimized for a very specific
purpose I don't want you to think that I
have a very pessimistic view of this
technology it is amazing technology and
will have tremendous positive impact
and every domain if you wanna talk
economics we will get free labor
physical cognitive so trillions of
dollars worth of labor it's a great help
to scientists it's useful for medical
research in any domain you can see a
positive impact from this technology
sadly this is a dual use technology and
so for every positive there is a
negative equivalent to it and this is my
last positive slide unfortunately so if
you're talking about free labor physical
cognitive you're talking about
unemployment unprecedented levels of
unemployment and you can maybe come up
with a social net safety system for
redistributing some of the income from
robots in the eyes but still it changes
changes society in very profound ways
what do you do with all that free time
how do you see your purpose in life for
many people their careers source of
pride and purpose likewise applications
of this technology to politics to
marketing to propaganda all will create
problems which we have not looked at
previously have not addressed in any
meaningful way and what's most
interesting on a previous slide there
was this listing typology of different
problems do we know about them can be
predicting the worst one is they're
known unknowns not being super
intelligent yourself you can't even
predict what things can happen and this
applies both to the positive side of it
it can be so good we can't even imagine
what its gonna do but also it could be
very bad so we're not sure what AI can
do for us or to us at the same time the
good news is there is a lot of interest
in this area and some very rich famous
powerful people have all expressed
concern about AI some about advanced AI
forms some about existing problems we
have with
algorithmic bias and unemployment but it
seems to be that this is becoming more
accepted five years ago ten years ago
you wouldn't see this happening it would
be purely science fiction and today
people realize that well I'm working on
this problem and I hope to succeed what
happens when I succeed so again we saw
it in a keynote address after all the
accomplishments elicited concerns are
brought up so this is becoming the new
norm and what is it they actually have
concern about we as people you can look
around the room a diverse group we have
different nationalities represented
different genders different geographies
but if you think about our biology our
DNA our brain structure we're almost
identical we can all fit in that pale
blue dot there right all human minds
human brains are variations are very
very small compared to the almost
infinite universe of possible Minds
possible brains so examples you can
think about immediately animal brains
very different preferences desires think
about science fiction alien Minds this
is kind of what we see now with software
we develop we can design it we can
evolve it we can randomly reduce it by
certain means but if it's not well
aligned to understand what we care about
what we value it could be a big problem
because they can still be much more
capable at optimizing for whatever
result we're trying to get while at the
same time having absolutely no human
common sense so I call this problem the
singularity paradox where you have a
super capable optimizer but it's really
dumb it just doesn't get things and
misunderstands playing which in a
literal sense and there is a lot of
problems you get by analyzing what such
an optimizer can do then faced with very
human problems so what I do in my work I
try to predict
what types of problems we can see with
intelligent software and it's software
so things you expect like bad design
bugs in the code definitely become a
problem but there are also things
specific to AI so if we get to the level
of performance where it's like a human
engineer human scientist it's working
and self-improvement it's designing the
next version of the software so it's
learning in its own it's making
modifications it's possible that it will
introduce improvements which you are not
happy about the worst problem as I see
it is malevolent purposeful design of
such systems we saw it mentioned but
really any type of AI system can be used
for both purposes it's a dual use
technology and there is very little
research on malevolent use of AI in fact
for a while there was absolutely nothing
we had one paper come out with a
workshop now there is a few papers
people working on it but it's a very new
area and there is a lot of problems
being described in very few solutions
how do you make it safe if it's an
insider threat from a human how do you
make it safe if the system itself is
independently learning and improving
worst part is that all the previous
problems can still be a part of us
malevolent design you can still screw up
your design you can still make mistakes
in your code we see it with computer
viruses all right so essentially what we
have same people who right now try to
design malware and make it as lethal as
damaging as possible would get use of
this very advanced technology advanced
capabilities and we try to understand
who who have is people who would be
possibly using this and the answer is
really everyone it's not just hackers
it's crazy people dumbs they calls
government's military everyone can find
beneficial use for this technology with
AI as a service you just add your own
goals right so it
trivial to do this you don't have to be
an expert the cost of doing this type of
attack gets greatly reduced if it used
to be that only a big government could
do something now with access to those
tools and individual hacker can do those
things likewise we need to understand
what can we possibly do with this
technology if we're talking about human
level AI or beyond there is really no
limit in fact we can't even predict all
the damage it could be trivial things
like taking over computational resources
terrorist acts any type of manipulation
for political purposes we saw it with
previous presidential election in u.s.
financial manipulation of a stock market
I want to slow down for a second and
maybe not concentrate as much on the
futuristic possibilities but talk about
what is possible today with today's
technology and you'll see some of it
kinda repeats the keynotes you heard
before that's a good sign we're getting
independent confirmation for importance
of it but think about how those
technologies developed for very good
purposes can be used for problematic
issues so you probably seen this
progression of our ability to generate
fake data deep fakes right we started
with really low resolution
black-and-white images in a few years we
got to you can't really tell it's not a
real image and now 2019 you can generate
the own individual examples if you go to
this person does not exist that calm I
mean it's a fun game where you can kind
of try and detect artifacts still but
it's becoming harder and harder for non
professionals to do so interesting
examples I'm very concerned with so
historically we always developed there's
adversarial inputs for artificial neural
networks but they are inspired modeled
on natural neural networks human brain
can we do the same with with humans can
we design images which fool humans are
just images any type of media so there
is a great example of this CAD dog
illusion we're taking an image modifying
just a few pixels they're able to fool
many humans into changing the labels of
the image and this is early work think
about how good this could get the same
progression continues another example is
for fooling machines again just a few
pixel differences in the first case a
guy's confident this is a male face
almost 100 percent confidence looks the
same to me
opposite confidence with two pixels flip
how can this be used for
malevolent purposes well if you can
create fake video
fake sound fake pictures what do you
believe if a political video surface is
showing you candidates saying really
stupid things
is it real president a fake one how
would you know it becomes a problem if
you have a phone call coming from your
friend your boss can you can you tell
who they are we're starting to see some
legal solutions being proposed in terms
of governments so for example in
California they passed a law saying a
machine has to self identify as a
machine cannot pretend to be a human
cannot fake being a person if it's a bad
but making something illegal doesn't
always fix that problem right computer
viruses are not legal spam is not legal
but it doesn't seem to reduce the amount
of problems we get from that
another interesting set of problems
comes specifically from machine learning
I mentioned that those artificial neural
networks are inspired by the human model
natural on your own networks and I often
say that children are untrained neural
networks deployed in real data right
that's what we do we take baby humans
with Rome in the real world we see how
they fail and if you are parent if you
have kids you know the typical failure
modes you can predict them and it's
interesting if you look at the latest
research on misalignment of machine
learning systems they use very technical
terms they have to make it
formal but at the end of the day there
is almost almost perfect mapping between
how kids grow up and what the machine
learning systems do and this helps you
predict how they can fail so things like
stealing the reward Channel seems
complicated but a kid gets into the
cookie jar right they supposed to get
rewarded for your behavior well they
just find the jar and they start you get
the same problems with with artificial
neural networks and it's nice because if
we understand how they can fail if we
study those problems we can predict what
you can anticipate with your product
with your service and later on I'm gonna
talk specifically about AI accidents but
this is one type of way to analyze this
space for what to expect so the good
news is there is more and more interest
in research in this area
boström super intelligence book was
mentioned there are other books now
specifically addressing issues with
every single subdomain you can think
about political manipulation economic
deep fakes fall of it there is now
healthy research going on people are
very interested top universities so if
you think about Oxford Cambridge MIT
Berkeley University of Louisville have
research groups devoted to AI safety
there are now governance panel at United
Nations at world government summit all
trying to figure out how do we control
this technology what do we want them to
do what is the actual solution to this
set of problems
early on then we were just starting this
work we wanted to understand well what
is the state of the art in AI safety
what have people proposed and so we did
this survey paper it's publicly
available you can grab it looking at
about 300 references everything we could
find going back to 1863 let's let's how
soon people started realizing okay there
could be could be issues and we try to
get all the data classified analyzed
just so we understand where we stand how
can we move forward if we don't have
good historical base for understanding
the set of proposed solutions and I
would guess if I was to ask you to come
up with some solutions
whatever you say would be an and one of
those lists one of those sub references
so I'll give you some examples I don't
have time for 300 of them but I'll give
you a feel for what people have
suggested as possible solutions here's
the first one I don't have a picture for
it basically machines if they smart
don't be nice to us because nice people
are smart people are nice right the same
logic or another one will never get
there we'll just never succeed so we
don't have to worry about it the worst
part about the solution is you can get
funding for it it's terrible I tried you
like that how many of you recognize this
guy doctor Kaczynski famous
mathematician in his manifesto he talks
about how big of a problem it is how
serious he takes it serious enough to
his solution is to kill computer
scientists for reasons of personal bias
I disagree with that solution as well
but it's important to realize that this
is the level of importance some people
place and this problem just in context
other solutions talk about integration
economic integration legal integration
so if we just applied the legal system
and financial rewards to robots it would
work out great they would all get
minimum wage jobs and be happy living
along us the problem is you can't really
apply human concepts of reward and
Punishment law financial reward to
non-human entities there are lots of
issues issues with copying itself so if
you can punish human permanently it's
not always the case with the eyes there
are issues with just
resources necessary and so on so while
it has been suggested I don't think it's
one of the better options for us to
drive while in a short term with
narrower ice having legal computable
language for smart contracts is very
important my my concern with all those
is always do they scale to more
intelligent systems can we start with
them and grow as AI grows and I think in
this case the answer is no another
solution is to say well the machines are
smarter than us how can we compete we
need to become smarter we need to
self-improve to the levels where we are
competitive and we can do it either
through some sort of a hybrid system
brain computer interface give you more
memory access to internet or just
straight-up upload your consciousness
into a machine in some sort of a
software simulation of you which would
run at a faster clock speed and be
competitive those are interesting
solutions and it's possible the
technology will get there the problem I
see is that we are not really solving
anything if we are plotting humanity
into a computer and we all become pieces
of software AI itself it seems like we
addressing our own problem there you may
disagree with this and feel very
comfortable with becoming a piece of
software but we need to discuss it a
little more probably the most famous
solution I'm sure all of you heard about
it at some point is Three Laws of
Robotics by Isaac Asimov right we all
know it is important to remember if you
read the books they fail every single
time
there are literary tools for adding
interesting books we are not meant to
actually solve anything there you'll
defined contradictory and if you
increase the number of laws to let's say
10 still doesn't solve anything right so
remember this is not an actual solution
the things which have some promise and
might scale but are still very much open
open problems software verification
formal verification where you create
mathematical proof of correctness it is
an expensive process but we know how to
do it for mission critical software
problem is we have no idea how to do it
for software capable of self improvement
learning working in novel domains this
is great for deterministic processes is
it doing what I'm telling it in step one
check if it's doing novel things we
don't know how to verify that right
another project we're working on is this
idea what people frequently like calling
putting a guy in prison I don't like
that term too much but that's what it is
then you studying computer viruses for
example you put it an air-gapped
computer isolated from the internet you
study inputs outputs what cause it makes
you trying to understand how it works
here the idea is similar plus you adding
additional limit and communication to
limit social engineering attacks right
so you can decide exactly what type of
data goes in the system can learn from
and what type of outputs can go out and
you can change safety levels based on
that so it seems like it would be useful
for anyone developing advanced AI
systems to have this playground his
sandbox for for experimenting with it so
as scientists we typically hate being
told what to do but we somehow all
agreed that certain things are unethical
and we shouldn't be doing them
biological weapons chemical weapons
experimenting and babies seem like
they're bad things and no one should be
doing it no matter how awesome it might
sound to enforce that we have especially
in academia the concept of review boards
you want to do some experiments and
people and animals you have to get
permission explaining how it's super
harmless or whatever harm is cost will
be justified by great results you need
to get it approved then we propose doing
this for software that sounded a little
crazy
who needs permission to write software
well since then many companies including
top
like google deepmind have added ethics
review boards
you probably heard any news where it's
an hour one recently and that didn't
work out as well but the other one the
secret one works well so what is the
purpose of this review boards some
software is completely benign phew
riding a calculator it's super
intelligent that basic algebra there is
no concerns of any kind go ahead if
you're developing a software which has
potential of self improvement learning
cross domain knowledge transfer
essentially it's a general intelligence
and if you add more compute it becomes
proportionately more powerful then maybe
you need someone else to kind of double
check with you are you sure you know
what you're doing do you have safety
mechanisms in place so this seems to be
the case more and more and again with
like open AI recent work on generating
natural language we saw this at least if
not limitation in not doing the project
but self limitation and providing the
model to everyone which is a good trend
I will conclude by showing you this
exponential chart you usually see charts
like that for AI accomplishments right
they get better and better this is a set
of accidents a yeah accidents going back
to the 50s this stops a 2016 we just
published a paper going to 2018 and I
have another 50 I have to add for 2019
it's growing exponentially it's becoming
more common as more and more people use
this technology and it seems like the
impact is becoming more severe so we can
pretty much make those predictions about
the future of this technology and if you
give me a specific device of product I
can probably tell you how it's going to
fail so this is something of great
interest to a lot of companies
partnership and he is collaborating with
us to to make this public database
accessible to all partners you can
submit your own examples if you did
something and it failed spec
ocularly surprisingly let us know we'll
add it to the collection but this is the
general trend and I think we succeeded
in identifying a lot of problems and not
so many solutions and it's exactly the
state of the art if you're interested in
working on something with a lot of
impact this is an area where we need
lots of smart people lots of funding
lots of resources if you want to learn
more all the papers behind those topics
are available you can go to my google
scholar account grab all of them if you
can find something don't have access
email me I'll send it to you and I think
I have time for questions
easy questions thank you I don't know if
a recording you want to use the mic what
what have you seen in terms of us
developing a is that our that could be
like good good for human a eyes like
protector a eyes that we can have us
like assistants or someone that helped
us combat a eyes that are harmful sort
of like an arms race of AI right so we
are seeing an arms race especially in
cybersecurity there is automated tools
for finding zero-day exploits and there
are tools for finding them and patching
them the problem is it's always dual use
technology if you develop a really good
bodyguard I can turn that bodyguard into
a really good killer just by flipping a
few bits right it's essentially the same
utility function just reversed so how do
we make sure that it's not dual use
application and progress is impressive
in many areas but it's always before you
release it you're not sure how it's
going to be used
we had some keynotes about Facebook
today I'm sure they weren't planning on
changing democracy to other systems then
they started uploading pictures of
college students so you have to you have
to invest a lot of time and trying to
understand side-effects from this
technology and
we're usually very bad at predicting it
and that's why I talk and two slides at
least about unknown unknowns you can
think about everything you can think
about what you see but if a system is
smarter than you it will find ways you
didn't consider that's what makes it
worse not all at once can throw yourself
Mike please might be a little
controversial but our different
countries approaching this differently
because the way I look at it data for
example China they say they're oh they
already have more data and what their
surveillance apparatus a lot more
advanced than what we have in the u.s.
they already have an advantage so over
here if you talk about it just from a
u.s. perspective is there a clear
disadvantage for people who are actually
looking at the safety compared to other
countries that might not be thinking
about it that way so in the narrow term
with issues like privacy definitely
China has very different set of rules
what you can get away with there you
probably wouldn't be able to do here and
definitely not in European Union so in
that regard they have better chance of
developing this technology but I'm
looking at it from the safety point of
view being the first one to create a
killer robot means you're going to be
killed first it doesn't give you an
advantage in terms of controlling it
properly and we are just now starting to
have good outreach in terms of AI safety
collaborations with our Chinese partners
there are some existing conferences
which are now adding workshops in that
issue and hopefully there is going to be
more collaboration and finding a unified
vision for what it is we want this
technology to accomplish for Humanity as
a whole not just okay we have a
communist AI will have a capitalist area
and they fight it out yeah
really good talk thank you so much
so right now you you mentioned dual use
right and I know in weapons of mass
destruction you know the United States
government the UN they have they have
responsibilities and priorities in place
to monitor dual use technology when it
comes to nuclear weapons chemical
weapons etc what can we take away from
that and what kind of challenges are
there when you you know transpose that
into AI and technology so specifically
would weaponize the AI and military
robotics there is a lot of work at UN
level there is what is called campaign
to stop killer robots and their whole
job is to get as many countries as they
can to sign sign the treaty saying we're
not going to develop this technology
we're not going to deploy it will always
have human-in-the-loop controlling those
systems but as always the case those who
have more advanced technology are not
very interested in signing it so
countries which don't have computers are
happy to sign it but countries which are
at the forefront which actually matter
and not as happy so that's where we need
a lot of political work governance
influence that to accomplish that
because that's the easiest to understand
in terms of damage technology so you
made a killer robot you unleash it it
gets hacked or something and nobody is
responsible if we can get that to be
properly controlled and governed there
is a good chance we might understand
some other concerns which are more
advanced and not as easy to explain to
your local politician if you have any
individual questions I'll stick around
thank you so much
[Applause]