```python
from IPython.display import HTML
from IPython.display import display

display(HTML("<style>.container { width:70% !important; }</style>"))
```

```python
%matplotlib inline
import numpy as np, scipy, scipy.stats as stats, pandas as pd, matplotlib.pyplot as plt, seaborn as sns
import statsmodels, statsmodels.api as sm
import sympy, sympy.stats
import pymc3 as pm
import daft

pd.set_option('display.max_columns', 500)
pd.set_option('display.width', 1000)
# pd.set_option('display.float_format', lambda x: '%.2f' % x)
np.set_printoptions(edgeitems=10)
np.set_printoptions(linewidth=1000)
np.set_printoptions(suppress=True)
np.core.arrayprint._line_width = 180

SEED = 42
np.random.seed(SEED)

sns.set()

import warnings
warnings.filterwarnings("ignore")
```

This blog post is part of the [Series: Monte Carlo Methods](https://weisser-zwerg.dev/posts/series-monte-carlo-methods/).

You can find this blog post on [weisser-zwerg.dev](https://weisser-zwerg.dev/posts/monte-carlo-fundamental-concepts/) or on [github](https://github.com/cs224/blog-series-monte-carlo-methods) as either [html](https://rawcdn.githack.com/cs224/blog-series-monte-carlo-methods/main/0010-fundamental-concepts.html) or via [nbviewer](https://nbviewer.jupyter.org/github/cs224/blog-series-monte-carlo-methods/blob/main/0010-fundamental-concepts.ipynb?flush_cache=true).

# Monte Carlo Fundamental Concepts

Table of Contents

1  Boundary between statistics and machine learning / AI

2  Monte Carlo sampling methodology

2.1  A word of caution about the picture of "Bayes updating"

3  Statistical Models

3.1  Quality of a model:  model comparison, model selection, model averaging

4  How to construct models: keep the Bayes' theorem and the law of total probability (extending the conversation) always in mind

4.1  Bayes' theorem

4.2  Law of total probability (extending the conversation)

5  What is the "right" representation of (multi-dimensional) probability distributions?

6  Operations on probability distributions: marginalization and conditional probability

7  Calculations with random variables: convolutions and other nasty stuff

8  Density Estimation

9  The algorithm has to always perform the same heavy lifting: there is no ancestral sampling

10  Appendix

10.1  Examples of mixed discrete continuous probability functions

10.1.1  Two independent coin tosses

10.1.2  Two component mixture model

## Boundary between statistics and machine learning / AI

Only very recently I found the work of 2010 [Turing Award](https://en.wikipedia.org/wiki/Turing_Award) winner [Leslie Valiant](https://en.wikipedia.org/wiki/Leslie_Valiant). In his book [Probably Approximately Correct](https://www.amazon.com/Probably-Approximately-Correct-Algorithms-Prospering/dp/0465060722/) he defines the two concepts **theoryful** and **theoryless**. This would be the place where I would draw the line between statistics and machine learning.

Theoryful: a human has to provide a model and there may be some data that helps to tune the model. A good example would be Newtonian motion of the planets around the sun where Newton's laws provide the model and the data helps to pinpoint the [gravitational constant](https://en.wikipedia.org/wiki/Gravitational_constant).

Theoryless: you take a very generic algorithm and a lot of data and the algorithm figures out the rest. A good example would be a [multilayer perceptron](https://en.wikipedia.org/wiki/Multilayer_perceptron) and pictures of objects, where the algorithm learns via data to identify the objects. The same multilayer perceptron architecture could be used to pilot a self-driving car.

## Monte Carlo sampling methodology

The general Monte Carlo sampling methodology relies on 2 ingredients:
* The model (a total/joint probability density function)
* A sampling algorithm, where there are in principle only 2 core building blocks:
    * [Importance Sampling](https://en.wikipedia.org/wiki/Importance_sampling) and its variations / synonyms like Sequential Importance Sampling (SIS), Sequential Monte Carlo (SMC), Particle Filters, ...
    * [Markov chain Monte Carlo (MCMC)](https://en.wikipedia.org/wiki/Markov_chain_Monte_Carlo) and its variations like Metropolis-Hastings (MH), Hamiltonian (or Hybrid) Monte Carlo (HMC, NUTS), Gibbs sampling, ...

The construction of sampling algorithms based on fine-grained composable abstractions
([FCA](https://web.archive.org/web/20130117175652/http://blog.getprismatic.com/blog/2012/4/5/software-engineering-at-prismatic.html)) will be the main focus of this blog post series.

In the case of Importance Sampling, you could argue that there is not only a single model, but a sequence of related models that form a "bridge" from something simple to the target model of interest. But we will get there in the course of this blog post series.

### A word of caution about the picture of "Bayes updating"

Initially, when I started to read about the Bayesian method of statistics and its inference methodology, I often heard the term [Bayes updating](https://statswithr.github.io/book/the-basics-of-bayesian-statistics.html#bayes-updating). The picture that this terminology evoked in me was that there is a sequential process of taking a prior, a likelihood for a single event and then perform "Bayes updating" to receive the posterior, with that posterior becoming the new prior for the next data-point from where the process continuous until all data-items are consumed.

$
\displaystyle
\begin{array}{rcl}
d_i&\in&\{d_1, d_2, ..., d_N\}\\
p(d_1,\theta)&=&p_0(\theta)\cdot p(d_1\,|\,\theta)\\
p(\theta\,|\,d_1)&=&\displaystyle \frac{p(d_1,\theta)}{\int p(d_1,\theta)\,\mathrm{d}\theta}=p_1(\theta)\\
p(d_2,\theta)&=&p_1(\theta)\cdot p(d_2\,|\,\theta)\\
p(\theta\,|\,d_2)&=&\displaystyle \frac{p(d_2,\theta)}{\int p(d_2,\theta)\,\mathrm{d}\theta}=p_2(\theta)\\
&...&\\
&=&p_N(\theta)
\end{array}
$

While this picture is of course correct and is what you would do if you calculated the posterior manually by using (for example) [conjugate priors](https://en.wikipedia.org/wiki/Conjugate_prior), this picture prevented me from understanding what is going on in especially MCMC methods, but also in SMC methods and I found it much more useful to think of the model as the (immutable – without updating) total joint probability including all data-points, e.g.:

$
\displaystyle
\begin{array}{rcl}
p(d_1,d_2,...,d_N,\theta)&=&p_0(\theta)\cdot p(d_1\,|\,\theta)\cdot p(d_2\,|\,\theta)\cdot ... \cdot p(d_N\,|\,\theta)\\
\end{array}
$

That is all you need for the main Monte Carlo sampling methods, because they only depend on an unnormalized probability density function and if you take $p(d_1=d_1,d_2=d_2,...,d_N=d_N,\theta)$ (you simply take the total joint probability function and [partially apply](https://en.wikipedia.org/wiki/Partial_application) its data arguments so that you receive a function of the set of unknown parameters $\theta$ alone, e.g. $f(\theta)=p(d_1=d_1,d_2=d_2,...,d_N=d_N,\theta)$) then you are ready to go, because $f(\theta)\propto p(\theta\,|\,d_1=d_1,d_2=d_2,...,d_N=d_N)=\frac{f(\theta)}{Z}$ where $Z$ (the so called [partition function](https://en.wikipedia.org/wiki/Partition_function_(mathematics))) is independent from $\theta$ and therefore a constant for the concrete data-set you're looking at.

## Statistical Models

In (Bayesian) statistics a model is basically a **total** [probability mass function](https://en.wikipedia.org/wiki/Probability_mass_function) (PMF; in the discrete case) or a [probability density function](https://en.wikipedia.org/wiki/Probability_density_function) (PDF; in the continuous case). It comes often in the form of a [probabilistic graphical model](https://en.wikipedia.org/wiki/Graphical_model) (PGM; also see [Bayesian hierarchical model](https://en.wikipedia.org/wiki/Bayesian_hierarchical_modeling)). As an example, you see below the graphical representation of multiple coin tosses using the same coin:

```python
pgm = daft.PGM(grid_unit=3.0, node_unit=1.5)

pgm.add_node("theta", r"$\theta$", 0, 2)
pgm.add_node("d", r"$d_i$", 0, 1, observed=True)
pgm.add_plate([-0.5, 0.5, 1.0, 1.0], label=r"$i = 1, \cdots, N$")#, shift=-0.1

pgm.add_edge("theta", "d")

pgm.render(dpi=100);
```

In standard mathematical notation this would be:

$\displaystyle
\begin{array}{rcl}
p(\theta) &=& \mathrm{Beta}(\alpha=1,\beta=1) \\
p(X_i=d_i\,|\,\theta) &=& \mathrm{Bernoulli}(X_i=d_i;p=\theta) \\
\displaystyle p(X_1=d_1, X_2 = d_2, ..., X_n = d_n, \theta)&=& \displaystyle p(\theta) \cdot\prod_{i=1}^N p(X_i=d_i\,|\,\theta)
\end{array}
$

Where on the left hand side you have the total probability and on the right hand side you have the prior probability $p(\theta)$ (here given as a [Beta distribution](https://en.wikipedia.org/wiki/Beta_distribution)) multiplied by the [conditional probabilities](https://en.wikipedia.org/wiki/Conditional_probability) $p(X_i=d_i\,|\,\theta)$ (here given as a [Bernoulli distribution](https://en.wikipedia.org/wiki/Bernoulli_distribution)). The important point to remember is **YOU NEED THE TOTAL PROBABILITY FUNCTION**. How you get it does not matter. The total probability function is a **scalar function**, e.g. once you provide all arguments you get a scalar.

For pure continuous probability density functions and pure discrete probability mass functions this is straight forward. In the pure continuous case you typically have a function like a product of Gauss/Normal:
$f(x_1,x_2;\mu,\sigma)=\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{1}{2}\frac{(x_1-\mu)^2}{\sigma^2} }\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{1}{2}\frac{(x_2-\mu)^2}{\sigma^2} }$. In the pure discrete case, you get a multi-dimensional array like for two dice rolls:

```python
A = sympy.ones(6,6)/36
A
```

$\displaystyle \left[\begin{matrix}\frac{1}{36} & \frac{1}{36} & \frac{1}{36} & \frac{1}{36} & \frac{1}{36} & \frac{1}{36}\\\frac{1}{36} & \frac{1}{36} & \frac{1}{36} & \frac{1}{36} & \frac{1}{36} & \frac{1}{36}\\\frac{1}{36} & \frac{1}{36} & \frac{1}{36} & \frac{1}{36} & \frac{1}{36} & \frac{1}{36}\\\frac{1}{36} & \frac{1}{36} & \frac{1}{36} & \frac{1}{36} & \frac{1}{36} & \frac{1}{36}\\\frac{1}{36} & \frac{1}{36} & \frac{1}{36} & \frac{1}{36} & \frac{1}{36} & \frac{1}{36}\\\frac{1}{36} & \frac{1}{36} & \frac{1}{36} & \frac{1}{36} & \frac{1}{36} & \frac{1}{36}\end{matrix}\right]$

For mixed cases I often got confused and therefore I elaborated two examples in painstaking detail in the appendix:
* a model for [two independent coin tosses](#Two-independent-coin-tosses) and
* a [two-component mixture model](#Two-component-mixture-model) with two observations.

### Quality of a model:  model comparison, model selection, model averaging

While this topic is beyond the scope of this blog post series, I nevertheless wanted to mention it. Once you have several models that are supposed to describe the same observations how do you then decide which model is actually the "best" model?

The landmark rule is that a model is only as good as it is able to describe/predict future/unseen data-points. An approximation of that characteristic can be obtained via [cross-validation](https://en.wikipedia.org/wiki/Cross-validation_(statistics%29), where one variant is [leave-one-out cross-validation](https://en.wikipedia.org/wiki/Cross-validation_(statistics%29#Leave-one-out_cross-validation) (LOOCV). You can either apply this directly or use some approximation like an [information criterion](https://docs.displayr.com/wiki/Information_Criteria). In "[Statistical Rethinking](https://www.amazon.com/-/de/dp/036713991X): A Bayesian Course with Examples in R and STAN" in chapter 7. "Ulysses' Compass" Richard McElreath proposes to use [Pareto-Smoothed Importance Sampling Cross-Validation](https://arxiv.org/abs/1507.02646) (PSIS) or the [Widely Applicable Information Criterion](https://en.wikipedia.org/wiki/Watanabe%E2%80%93Akaike_information_criterion) (WAIC). He adds a word of caution, though: [Deviance](https://en.wikipedia.org/wiki/Deviance_(statistics%29) is an assessment of predictive accuracy, not of truth!

John Kruschke proposes in "[Doing Bayesian Data Analysis](https://www.amazon.com/-/de/dp/0124058884/): A Tutorial with R, JAGS, and Stan" to use the [Bayes factor](https://en.wikipedia.org/wiki/Bayes_factor) (see for example my earlier blog post: [Model Comparison via Bayes Factor](https://weisser-zwerg.dev/posts/model-comparison-via-bayes-factor/)) or to create a mixture model out of the different competing (sub-)models and then simply apply the machinery of the Monte Carlo sampling methodology to get probabilities for each sub-model as $p_1, p_2, ..., p_M$. The idea then is to say that the model with the highest probability describes the data best. This approach is also the basis for [model averaging](https://en.wikipedia.org/wiki/Ensemble_learning#Bayesian_model_averaging) to create a new and better model by simply using this mixture model for your predictions.

## How to construct models: keep the Bayes' theorem and the law of total probability (extending the conversation) always in mind

### Bayes' theorem

$
\displaystyle
p(x,y)=p(y|x)p(x)=p(x|y)p(y)
$

This is how you typically construct ([generative](https://en.wikipedia.org/wiki/Generative_model)) models. You basically say: if I knew $x$ then knowing $y$ would be simple. Let's take a physical example like throwing a ball. In a vacuum the trajectory of the ball would be a parabola. Under realistic conditions you have wind and other disturbances. But if you say, if I know where the ball measured by location $x$ (a 3 dimensional vector) was at $t=t_0$ then I could tell where the ball would be at $t=t_1$:

$
\displaystyle
\begin{array}{rcl}
x_1& = & x_0+v(t_1-t_0)+(g/2)(t_1-t_0)^2+\varepsilon_m;\quad\text{with }\varepsilon_m\sim\mathcal{N}(0,\sigma_m)\\
\Rightarrow p(x_1\,|\,x_0) & = &\mathcal{N}(x_1; x_0+v(t_1-t_0)+(g/2)(t_1-t_0)^2,\sigma_m)
\end{array}
$

If you then in addition say that at time $t_0$ the ball was roughly at $\mu_0+\varepsilon_l$ with $\varepsilon_l\sim\mathcal{N}(0,\sigma_l)$, e.g. $p(x_0) = \mathcal{N}(x_0; \mu_0,\sigma_l)$ then you can tell the probability distribution of $p(x_1)$ by:

$
\displaystyle
\begin{array}{rcl}
p(x_0,x_1) &=& p(x_0)\cdot p(x_1\,|\,x_0) = \mathcal{N}(x_0; \mu_0,\sigma_l)\cdot\mathcal{N}(x_1; x_0+v(t_1-t_0)+(g/2)(t_1-t_0)^2,\sigma_m)\\
p(x_1)&=&\int p(x_0,x_1)\mathrm{d}x_0
\end{array}
$

### Law of total probability (extending the conversation)

Initially, when I saw the [law of total probability](https://en.wikipedia.org/wiki/Law_of_total_probability) for the first time I neglected it, because it looks like redundant to the Bayes' theorem:

$
p(x)=\int p(x\,|\,y)\cdot p(y)\,\mathrm{d}y
$

I only learned to appreciate this law really when reading chapter 1.3.6 "The Law of Total Probability" in [Causal Inference in Statistics - A Primer](https://www.amazon.com/Causal-Inference-Statistics-Judea-Pearl/dp/1119186846) by Judea Pearl ([Turing Award](https://en.wikipedia.org/wiki/Turing_Award) winner 2011), Madelyn Glymour and Nicholas P. Jewell. There they also presented it under different synonymous names like "the law of alternatives" or "extending the conversation". In the book they call it "conditionalizing on y", but the term "extending the conversation" was what stuck with me.

Sometimes you just don't know how to get to $p(x)$, but if you knew $y$ you also would know how to get to $x$, e.g. you know $p(x\,|\,y)$. You're "extending the conversation" to include $y$ in addition to $x$. And often getting to a $p(y)$ is also possible.

It is basically the reverse idea from above in the example with the motion of the ball. If you just said I want to know $p(x_1)$ I would not know how to get there, but if you introduce $p(x_1\,|\,x_0)$ this opens up the path for a recursive calculation to the point where the ball was thrown and at that point you either know the position exactly, e.g. $p(x)=\delta(x = x_\text{start})$ (where the $\delta$ is the [Dirac delta](https://en.wikipedia.org/wiki/Dirac_delta_function)) or you give it some uncertainty, e.g. $p(x)=\mathcal{N}(x; x_\text{start},\sigma_l)$. Eitherway, you can construct a generative model.

## What is the "right" representation of (multi-dimensional) probability distributions?

That is another point that took me some time to digest. The names like probability-density-function (PDF) or probability-mass-function (PMF) suggest a mathematical form like a function. The typical example is the Gauss bell shape:

```python
plt.figure(figsize=(6, 3), dpi=80, facecolor='w', edgecolor='k')
ax = plt.subplot(1, 1, 1)
x = np.linspace(-3.0,3.0,100)
y = stats.norm(0,1).pdf(x)
ax.plot(x,y);
```

In this case the probability density function can be given as a mathematical function:

$\displaystyle
\frac{1}{\sqrt{2\pi}\sigma}e^{- \frac{\left(x - \mu\right)^{2}}{2 \sigma^{2}}}
$

But in most cases that is not possible. So what should you do then?

I cannot remember anymore in which source I've read it, but the author said that for probability distributions the "right" representation might actually be collections of samples.

Let's look at a toy example. We sample 1000 values from a normal distribution with mean 0 and standard deviation 1. Then we act as if we would only know that the samples are drawn from a normal distribution, but with unknown $\mu$ and $\sigma$. In PyMC3 this example looks like this:

```python
samples = stats.norm(0,1).rvs([1000])

with pm.Model() as model:
    mu = pm.Normal("mu", 0.0, 10.0)
    sigma = pm.Gamma('sigma', alpha=3.0, beta=1.0)
    x = pm.Normal("x", mu=mu, sigma=sigma, observed=samples)
    trace = pm.sample(return_inferencedata=True)
```

    Auto-assigning NUTS sampler...
    Initializing NUTS using jitter+adapt_diag...
    Multiprocess sampling (4 chains in 4 jobs)
    NUTS: [sigma, mu]

  <progress value='8000' class='' max='8000' style='width:300px; height:20px; vertical-align: middle;'></progress>
  100.00% [8000/8000 00:01<00:00 Sampling 4 chains, 0 divergences]

    Sampling 4 chains for 1_000 tune and 1_000 draw iterations (4_000 + 4_000 draws total) took 2 seconds.

Just as a reminder and to make the link to what we said above (about the model) more clear, let's be explicit and give the probability density function in sympy for two data-points instead of the 1000. All that PyMC3 does is to construct such a total joint probability density function and evaluate it at the given data-points. So in our example $x_1$ and $x_2$ are given and $\mu$ and $\sigma$ are "free" variables.

As another remark: the [Gamma distribution](https://en.wikipedia.org/wiki/Gamma_distribution) is sometimes parametrized via $\alpha, \beta$ and sometimes via $k, \theta$: $\alpha = k; \beta=1/\theta$

```python
smu, sx = sympy.symbols(r'\mu x')
sx1, sx2 = sympy.symbols(r'x1:3')
ssigma = sympy.Symbol("\sigma", positive=True)

prior_mu = sympy.stats.crv_types.NormalDistribution(0, 10).pdf(smu)
prior_sigma = sympy.stats.crv_types.GammaDistribution(3, 1).pdf(ssigma)
likelihood = sympy.stats.crv_types.NormalDistribution(smu, ssigma).pdf(sx1)*sympy.stats.crv_types.NormalDistribution(smu, ssigma).pdf(sx2)
total = sympy.UnevaluatedExpr(prior_mu)*sympy.UnevaluatedExpr(prior_sigma)*sympy.UnevaluatedExpr(likelihood)
sympy.Matrix([prior_mu, prior_sigma, likelihood])
```

$\displaystyle \left[\begin{matrix}\frac{\sqrt{2} e^{- \frac{\mu^{2}}{200}}}{20 \sqrt{\pi}}\\\frac{\sigma^{2} e^{- \sigma}}{2}\\\frac{e^{- \frac{\left(- \mu + x_{1}\right)^{2}}{2 \sigma^{2}}} e^{- \frac{\left(- \mu + x_{2}\right)^{2}}{2 \sigma^{2}}}}{2 \pi \sigma^{2}}\end{matrix}\right]$

```python
total
```

$\displaystyle \frac{\sqrt{2} e^{- \frac{\mu^{2}}{200}}}{20 \sqrt{\pi}} \frac{\sigma^{2} e^{- \sigma}}{2} \frac{e^{- \frac{\left(- \mu + x_{1}\right)^{2}}{2 \sigma^{2}}} e^{- \frac{\left(- \mu + x_{2}\right)^{2}}{2 \sigma^{2}}}}{2 \pi \sigma^{2}}$

PyMC3 created a trace object containing samples of the "free"/unknown variables. We can look at them as follows:

```python
trace_df = trace['posterior'].to_dataframe()
trace_df
```

      mu
      sigma

      chain
      draw

      0
      0
      -0.025535
      0.995429

      1
      -0.000475
      0.982951

      2
      0.036382
      0.960661

      3
      0.069528
      0.960479

      4
      0.052084
      0.957739

      ...
      ...
      ...
      ...

      3
      995
      0.014148
      1.015268

      996
      -0.010790
      0.948807

      997
      0.061342
      1.008478

      998
      0.041587
      1.006843

      999
      0.046756
      0.997424

4000 rows × 2 columns

If we would want to know $\mu$ and $\sigma$ we could take the mean:

```python
trace_df.mean().to_frame()
```

      0

      mu
      0.019798

      sigma
      0.981421

There is a built-in PyMC3 method that gives us more details in addition to the mean values:

```python
pm.summary(trace)
```

      mean
      sd
      hdi_3%
      hdi_97%
      mcse_mean
      mcse_sd
      ess_bulk
      ess_tail
      r_hat

      mu
      0.020
      0.031
      -0.038
      0.074
      0.001
      0.0
      3294.0
      3168.0
      1.0

      sigma
      0.981
      0.023
      0.941
      1.024
      0.000
      0.0
      3464.0
      2734.0
      1.0

The *hdi* stands for [Highest (Posterior) Density Interval](https://en.wikipedia.org/wiki/Credible_interval) and gives the range in which credible values are located. As you can see, our real values 0.0 for $\mu$ and 1.0 for $\sigma$ are within those hdi invervals.

To get a better idea of the plausible parameter values we'll have a look at a [density plot](https://www.statsmodels.org/stable/examples/notebooks/generated/kernel_density.html).

```python
# list(statsmodels.nonparametric.kde.kernel_switch.keys())
```

```python
def kdeplot(lds, parameter_name=None, x_min = None, x_max = None, ax=None, kernel='gau'):
    if parameter_name is None and isinstance(lds, pd.Series):
        parameter_name = lds.name
    kde = sm.nonparametric.KDEUnivariate(lds)
    kde.fit(kernel=kernel, fft=False, gridsize=2**10)
    if x_min is None:
        x_min = kde.support.min()
    else:
        x_min = min(kde.support.min(), x_min)
    if x_max is None:
        x_max = kde.support.max()
    else:
        x_max = max(kde.support.max(),  x_max)
    x = np.linspace(x_min, x_max,100)
    y = kde.evaluate(x)
    if ax is None:
        plt.figure(figsize=(6, 3), dpi=80, facecolor='w', edgecolor='k')
        ax = plt.subplot(1, 1, 1)
    ax.plot(x, y, lw=2)
    ax.set_xlabel(parameter_name)
    ax.set_ylabel('Density')
```

```python
kdeplot(trace_df['mu'], x_min=-0.5, x_max=0.5)
```

```python
kdeplot(trace_df['sigma'], x_min=0.75, x_max=1.2)
```

We will talk about it in more detail below, but what we did above is that we looked at the marginal likelihood of only $\mu$ or only $\sigma$. If your probability density function is given as samples then marginalizing out other parameters is trivial. You only take the parameter(s) (the columns in the sample dataframe) you're interested in.

It is important to understand that the samples, e.g. each row in the sample dataframe, are **jointly plausible**, e.g. they represent the joint probability distribution. If two parameters would be correlated you would see that in the pairsplot like below. In our example the two parameters are obviously uncorrelated.

```python
sns.pairplot(trace_df, height=3);
```

## Operations on probability distributions: marginalization and conditional probability

While the marginalization and conditional probability operations are quite involved when you have to deal with probability distributions in their mathematical functions form these operations are *trivial* if you have samples representing the probability distributions.

Let's start with marginalization. Let's assume you have a joint probability density function $p(x,y)$ for two random variables $X$ and $Y$. Then you marginalize out $y$ by integrating over all values of $y$:

$
\displaystyle
\begin{array}{rcl}
p(x)&=&\displaystyle\int p(x,y)\,\mathrm{d}y
\end{array}
$

This integration may be quite involved even in simple cases like a [multi-variate normal distribution](https://en.wikipedia.org/wiki/Multivariate_normal_distribution). Have a look at [Bayes’ Theorem for Gaussians](http://web4.cs.ucl.ac.uk/staff/C.Bracegirdle/bayesTheoremForGaussians.pdf) by Chris Bracegirdle for details.

In the case of samples, on the other hand, marginalization is *trivial*! You simply project (in the sense of an [SQL projection](https://en.wikipedia.org/wiki/Projection_%28relational_algebra%29)) on the column(s) you're interested in and you get the marginal distribution for that parameter set.

The conditional probability is defined as:

$
\displaystyle
\begin{array}{rcl}
p(x,y)&=&\displaystyle p(y\,|\, x)p(x) \\
\Rightarrow p(y\,|\, x=x_0)&=&\displaystyle \frac{p(x_0,y)}{p(x_0)}\\
&=&\displaystyle \frac{p(x=x_0,y)}{\int p(x=x_0,y)\mathrm{d}y} \text{ you simply normalize the function} f(y)=p(x=x_0,y)
\end{array}
$

You have to divide two distributions and often have to calculate integrals. I also show the variation where you use the marginalization $\int p(x=x_0,y)\mathrm{d}y$, because sometimes that's useful to remember. $p(x=x_0,y)$ is no probability distribution in $y$ any longer, because $\int p(x=x_0,y)\mathrm{d}y < 1$.

If you are in the world of samples then the conditional probability is calculated like an [SQL *where* clause](https://en.wikipedia.org/wiki/Relational_algebra#Selection_%28%CF%83%29), e.g. you sub-select the dataframe for values where $x=x_0$. For continuous variables this condition is nearly never true. Therefore you have to sub-select with a *where* condition like $x_0-\delta<x<x_0+\delta$. Finally you only keep the columns (SQL projection) of the variables that are on the left hand side of the conditional probability bar ($(\cdot\,|\,\cdot)$), e.g. $y$ in our example.

## Calculations with random variables: convolutions and other nasty stuff

Calculating with random variables when their distribution is given as mathematical functions is just painfully difficult. The book [The algebra of random variables](https://www.amazon.com/algebra-variables-probability-mathematical-statistics/dp/0471014060) by Melvine Dale Springer gives a detailed overview. But even the simplest example like adding two random variables like $X=X_1+X_2$ where $X_1\sim p_1(x_1)$ and $X_2\sim p_2(x_2)$ involves [convolutions](https://en.wikipedia.org/wiki/Convolution), e.g. $X\sim p(x)= \int p_1(x-x_2)p_2(x_2)\,\mathrm{d}x_2$. The library [PaCal](https://github.com/jszymon/pacal) (ProbAbilistic Calculator) or [SymPy](https://docs.sympy.org/latest/modules/stats.html) may help. Nevertheless it is just best to avoid these intricate calculations completely.

Luckily, in the world of samples, all of that is simple again! Imagine you have a distribution given as samples from $x_1$ and $x_2$. You then construct samples of $x_1+x_2$ by simply adding them up for every row in your samples dataframe, e.g. you create a new column $x$ by row-wise calculating $x = x_1 + x_2$. And there you are, you have samples of the random variable $X$!

Another example might be like above, like [when we calculated the trajectory of the ball](#Bayes'-theorem). You could also see that example as $X_1 = X_0 + \Delta X$, where $\Delta X$ would be a distribution depending on the time delta $t_2-t_1$, the velocity $v$ and the acceleration $g$. In such cases it is typically easiest to see one of the variables as the starting point, e.g. $X_0$ and then use the law of total probability to construct $p(x_0, x_1)=p(x_0)p(x_1\,|\,x_0)$ and then calculate on the "conditional" side of $p(x_1\,|\,x_0)$, e.g.

$
\displaystyle
p(x_1\,|\,x_0) = \mathcal{N}(x_1; x_0+v(t_1-t_0)+(g/2)(t_1-t_0)^2,\sigma_m)
$

As you can see, on the "conditional" side of $p(x_1\,|\,x_0)$ (the side involving $x_0$) we simply calculate like normal! We simply add, subtract, multiply and whatever else we want to do. We also avoid the convolution by sampling from the total joint probability distribution $p(x_0, x_1)$ and marginalizing on the samples by simply dropping the $x_0$ column if we really wanted to. **No convolutions and no other difficult operations on distributions**! Just normal math and sampling on the extended joint space.

## Density Estimation

In the past, whenever I was dealing with machine learning methods, I always felt disappointed; disappointed about the lack of "insight" and "understanding" and that machine learning methods only give me a uni-directional way of reasoning.

Let's take regression or classification for example. Even the recent really impressive success stories like [object detection](https://en.wikipedia.org/wiki/Object_detection) and [image segmentation](https://en.wikipedia.org/wiki/Image_segmentation) only give you a way from an input to an output. If you wanted to know the most distinguishing features of a dog vs. a cat you're out of luck (there may be ways, but no straight-forward ones).

In my first encounter with [Bayesian networks](https://en.wikipedia.org/wiki/Bayesian_network) I was so impressed with the fact that you can not only reason forward from disease to symptoms, but also backward like given the symptoms of a patient, what are the most likely root causes, e.g. which disease might be the root cause and which actions/drugs might be the best way forward.

It took me quite some time to understand "what I really wanted". I wanted "all the answers", not only partial answers. And again some more time went by until I found the "term" that refers to what I want: [density estimation](https://en.wikipedia.org/wiki/Density_estimation). This term is often accompanied by other terms like [unsupervised learning](https://en.wikipedia.org/wiki/Unsupervised_learning) and recent advances in machine learning are also targeting density estimation directly via [deep generative models](https://en.wikipedia.org/wiki/Generative_model#Deep_generative_models). In the end it all boils down to "discovering" the [causal network](https://en.wikipedia.org/wiki/Causal_model) of the world, so that in the end you can answer any question you'd come up with, not only the ones the designer of the "system" wanted you to ask.

## The algorithm has to always perform the same heavy lifting: there is no ancestral sampling

When I read the paper [Mixed Hamiltonian Monte Carlo for Mixed Discrete and Continuous Variables](https://arxiv.org/abs/1909.04852) by Guangyao Zhou he used an exmaple of a [24 dimensional Gaussian Mixture Model](https://github.com/StannisZhou/mixed_hmc/blob/master/momentum/potential/simple_gmm.py) **without observations** to show the power of his approach. Initially, I felt as if this was cheating, because in the end [forward/ancestral](https://ermongroup.github.io/cs228-notes/inference/sampling) sampling is trival in that case. I had to remind myself that the MCMC algorithm he invented has to perform the same heavy lifting in the given example without observations as it has to perform in a situation that involves observations and the same number of free variables. The probabilistic programming languages and frameworks like [Stan](https://mc-stan.org/) or [PyMC3](https://docs.pymc.io/) do not employ any additional intelligence to decide which method might be most appropriate in which case.

> You always take a model in the form of a scalar valued function representing the (unnormalized) total joint probability function and then you always apply the same hammer like an [SMC](https://www.amazon.com/-/de/dp-0387951466/dp/0387951466/) or [MCMC](https://www.amazon.com/-/de/dp-B008GXJVF8/dp/B008GXJVF8/) algorithm.

If you want to be smart on that level you have to do it yourself as a human with your own brain.

## Appendix

### Examples of mixed discrete continuous probability functions

Below I am exaggerating the level of explicitness at each step of the calculation and the level to which I use [SymPy](https://www.sympy.org/en/index.html) to achieve maximum clarity. I hope that this will help you to avoid a similar level of confusion that I found myself in. In addition you might learn to appreciate to use [SymPy](https://www.sympy.org/en/index.html), which is really great for symbolic algebra.

#### Two independent coin tosses

The first example is about two independent coin tosses where we don't know $\theta\in[0,1]$, the probability for heads coming up. The total joint probability function (I only say "probability function" and avoid the terms *densite* or *mass*, because in the mixed case it is not clear in which situation we are) is then given by: $p(d_1,d_2,\theta)=p(X_0=d_0\,|\,\theta)\cdot p(X_1=d_1\,|\,\theta)\cdot p(\theta)$. If we take $p(\theta)=\mathrm{Beta}(1,1)=1$ for $\theta\in[0,1]$ then we get for each coin a one dimensional matrix with two entries:

```python
stheta = sympy.var(r'\theta')
D1 = sympy.Matrix([[(1-stheta)], [stheta]])
D1
```

$\displaystyle \left[\begin{matrix}1 - \theta\\\theta\end{matrix}\right]$

The entry with index 0 belongs to the case where we observe "tail" and the entry with index 1 belongs to the case where we observe "head". We can index into that vector like so:

```python
display(D1[0]),display(D1[1]);
```

$\displaystyle 1 - \theta$

$\displaystyle \theta$

And the indexing operation can be seen as a function that takes an integer and returns a scalar value. Exactly what we need for a probability function.

For the second coin we take a 2x1 matrix (a horizontal lying vector), so that when we multiply the two we will get a 2x2 matrix for the 2x2 combinations of tail and head for each coin:

```python
D2 = sympy.Matrix([[(1-stheta), stheta]])
D2
```

$\displaystyle \left[\begin{matrix}1 - \theta & \theta\end{matrix}\right]$

```python
D = D1*D2
D
```

$\displaystyle \left[\begin{matrix}\left(1 - \theta\right)^{2} & \theta \left(1 - \theta\right)\\\theta \left(1 - \theta\right) & \theta^{2}\end{matrix}\right]$

We said above that a probability function needs to be a function that takes as arguments the values of all random variables and delivers a scalar. In python this would look like:

```python
def two_coin_tosses_probability_function(theta, d1, d2):
    v = D.subs(stheta, theta)
    v = v[d1,d2] # we use indexing to access a concrete slot in the multi dimensional array
    return v
```

If we then wanted to know the probability of one head and one tail for a fair coin we would get:

```python
theta = sympy.Integer(1)/2
display(theta)
two_coin_tosses_probability_function(theta, 1, 0)
```

$\displaystyle \frac{1}{2}$

$\displaystyle \frac{1}{4}$

The other way we could have done this would be to use the [Kronecker delta](https://en.wikipedia.org/wiki/Kronecker_delta) instead of vector/matrix indexing and using the fact that any number to the power of $0$ is $1$: $x^0=1 \quad\forall x\in \mathbb{R}\setminus 0$, e.g. we receive the neutral element for our product of probability functions.

```python
sd1, sd2 = sympy.symbols('d1:3') # we define two new sympy symbols for d1 and d2
sd1, sd2
```

    (d1, d2)

Below, the first $1$ is to remind us of the $p(\theta)=\mathrm{Beta}(1,1)=1$. In case that we used another prior this would be a different factor.

```python
D_ = 1*(1-stheta)**sympy.KroneckerDelta(0,sd1)*stheta**sympy.KroneckerDelta(1,sd1)*(1-stheta)**sympy.KroneckerDelta(0,sd2)*stheta**sympy.KroneckerDelta(1,sd2)
D_
```

$\displaystyle \theta^{\delta_{1 d_{1}}} \theta^{\delta_{1 d_{2}}} \left(1 - \theta\right)^{\delta_{0 d_{1}}} \left(1 - \theta\right)^{\delta_{0 d_{2}}}$

```python
def two_coin_tosses_probability_function_(theta, d1, d2):
    v = D_.subs(stheta, theta)
    v = v.subs(sd1,d1)
    v = v.subs(sd2,d2)
    return v
```

And of course you get the same result:

```python
two_coin_tosses_probability_function(theta, 1, 0)
```

$\displaystyle \frac{1}{4}$

The Kronecker delta variant looks more like a function, but a computer works better with an index lookup in a matrix. Finally we could use an **if-then-else** structure, but many probabilistic programming environments behave clumsy when it comes to if-then-else constructs.

#### Two component mixture model

The second example is a two component mixture model. In most cases people will show a mixture of two Gaussians, but with two Gaussians you can get away with using an array of $\mu$ and $\sigma$. Let's look at mixing one Gaussian with one Exponential distribution:

```python
smu, sx = sympy.symbols(r'\mu x')
sx1, sx2 = sympy.symbols(r'x1:3')
si, sj = sympy.symbols('i j', integer=True)
sp0, sp1 = sympy.symbols(r'p:2')
ssigma = sympy.Symbol("\sigma", positive=True)
slambda = sympy.Symbol("\lambda", positive=True)
sympy.Matrix([smu, ssigma, slambda, sx, sx1, sx2, si, sj, sp0, sp1]).T # just for displaying the symbols
```

$\displaystyle \left[\begin{matrix}\mu & \sigma & \lambda & x & x_{1} & x_{2} & i & j & p_{0} & p_{1}\end{matrix}\right]$

```python
sympy.stats.crv_types.NormalDistribution(smu, ssigma).pdf(sx)
```

$\displaystyle \frac{\sqrt{2} e^{- \frac{\left(- \mu + x\right)^{2}}{2 \sigma^{2}}}}{2 \sqrt{\pi} \sigma}$

```python
sympy.stats.crv_types.ExponentialDistribution(slambda).pdf(sx)
```

$\displaystyle \lambda e^{- \lambda x}$

For the mixture to work we use a mixture indicator $i\in\{0,1\}$. If $i$ is 0 then we draw from the Exponential distribution and when $i$ is 1 we draw from the normal distribution.

```python
sympy.stats.frv_types.BernoulliDistribution(sp1, 1, 0).pmf(si)
```

$\displaystyle \begin{cases} p_{1} & \text{for}\: i = 1 \\1 - p_{1} & \text{for}\: i = 0 \\0 & \text{otherwise} \end{cases}$

```python
# print(sympy.latex(sympy.stats.frv_types.BernoulliDistribution(sp1, 1, 0).pmf(sx)))
```

```python
sympy.stats.frv_types.BernoulliDistribution(sp1, 1, 0).pmf(1)
```

$\displaystyle p_{1}$

```python
sympy.stats.frv_types.BernoulliDistribution(sp1, 1, 0).pmf(0)
```

$\displaystyle 1 - p_{1}$

For a single observation the total joint probability function would look as follows:

$
\displaystyle
p(p_1,i,\lambda,\mu,\sigma,x)=p(p_1)p(i\,|\,p_1)\begin{cases}p(\lambda)p(x\,|\,\lambda) & \text{for}\: i = 0\\ p(\mu)p(\sigma)p(x\,|\,\mu,\sigma) & \text{for}\: i = 1 \\0 & \text{otherwise} \end{cases}
$

Below we're using SymPy to construct the total joint probability function for two observations:

```python
sprior_exponential = sympy.stats.crv_types.ExponentialDistribution(1).pdf(slambda)
sprior_normal = sympy.stats.crv_types.ExponentialDistribution(1).pdf(ssigma)*sympy.stats.crv_types.NormalDistribution(0, 100).pdf(smu)
sprior_mixture_components = sprior_normal*sympy.stats.crv_types.BetaDistribution(1,1).pdf(sp1)
sprior = sprior_exponential*sprior_normal*sprior_mixture_components
smixture_likelihood = \
    (sympy.stats.frv_types.BernoulliDistribution(sp1, 1, 0).pmf(0)*sympy.stats.crv_types.ExponentialDistribution(slambda).pdf(sx1))**sympy.KroneckerDelta(0,si) * \
    (sympy.stats.frv_types.BernoulliDistribution(sp1, 1, 0).pmf(1)*sympy.stats.crv_types.NormalDistribution(smu, ssigma).pdf(sx1))**sympy.KroneckerDelta(1,si) * \
    (sympy.stats.frv_types.BernoulliDistribution(sp1, 1, 0).pmf(0)*sympy.stats.crv_types.ExponentialDistribution(slambda).pdf(sx2))**sympy.KroneckerDelta(0,sj) * \
    (sympy.stats.frv_types.BernoulliDistribution(sp1, 1, 0).pmf(1)*sympy.stats.crv_types.NormalDistribution(smu, ssigma).pdf(sx2))**sympy.KroneckerDelta(1,sj)
smixture = sprior * smixture_likelihood
smixture
```

$\displaystyle \frac{\left(\lambda \left(1 - p_{1}\right) e^{- \lambda x_{1}}\right)^{\delta_{0 i}} \left(\lambda \left(1 - p_{1}\right) e^{- \lambda x_{2}}\right)^{\delta_{0 j}} \left(\frac{\sqrt{2} p_{1} e^{- \frac{\left(- \mu + x_{1}\right)^{2}}{2 \sigma^{2}}}}{2 \sqrt{\pi} \sigma}\right)^{\delta_{1 i}} \left(\frac{\sqrt{2} p_{1} e^{- \frac{\left(- \mu + x_{2}\right)^{2}}{2 \sigma^{2}}}}{2 \sqrt{\pi} \sigma}\right)^{\delta_{1 j}} e^{- \lambda} e^{- \frac{\mu^{2}}{10000}} e^{- 2 \sigma}}{20000 \pi}$

Often, when you ask how to model a mixture model in a probabilistic programming system like Stan or PyMC3, people will recommend that you marginalize out the discrete variables like $i$ and $j$ above. Systems like Stan cannot deal with discrete free random variables at all. PyMC3 can deal with them, but sampling will be slow and hard, so that this recommendation actually makes sense.

Let's use SymPy to marginalize out the discrete variables $i$ and $j$. Marginalizing out discrete variables means summing over them:

```python
smixture_marginal_1 = sympy.summation(smixture, (si, 0, 1), (sj, 0, 1))
smixture_marginal_1
```

$\displaystyle \frac{\left(\lambda \left(1 - p_{1}\right) e^{- \lambda x_{1}} + \frac{\sqrt{2} p_{1} e^{- \frac{\left(- \mu + x_{1}\right)^{2}}{2 \sigma^{2}}}}{2 \sqrt{\pi} \sigma}\right) \left(\lambda \left(1 - p_{1}\right) e^{- \lambda x_{2}} + \frac{\sqrt{2} p_{1} e^{- \frac{\left(- \mu + x_{2}\right)^{2}}{2 \sigma^{2}}}}{2 \sqrt{\pi} \sigma}\right) e^{- \lambda} e^{- \frac{\mu^{2}}{10000}} e^{- 2 \sigma}}{20000 \pi}$

For clarity we may only look at the likelihood part without the prior components:

```python
sympy.summation(smixture_likelihood, (si, 0, 1), (sj, 0, 1))
```

$\displaystyle \left(\lambda \left(1 - p_{1}\right) e^{- \lambda x_{1}} + \frac{\sqrt{2} p_{1} e^{- \frac{\left(- \mu + x_{1}\right)^{2}}{2 \sigma^{2}}}}{2 \sqrt{\pi} \sigma}\right) \left(\lambda \left(1 - p_{1}\right) e^{- \lambda x_{2}} + \frac{\sqrt{2} p_{1} e^{- \frac{\left(- \mu + x_{2}\right)^{2}}{2 \sigma^{2}}}}{2 \sqrt{\pi} \sigma}\right)$

Obviously, we can do the same in the matrix picture rather than using Kronecker deltas:

```python
M1 = sympy.Matrix([
    sympy.stats.frv_types.BernoulliDistribution(sp1, 1, 0).pmf(0)*sympy.stats.crv_types.ExponentialDistribution(slambda).pdf(sx1),
    sympy.stats.frv_types.BernoulliDistribution(sp1, 1, 0).pmf(1)*sympy.stats.crv_types.NormalDistribution(smu, ssigma).pdf(sx1)])
M2 = sympy.Matrix([[
    sympy.stats.frv_types.BernoulliDistribution(sp1, 1, 0).pmf(0)*sympy.stats.crv_types.ExponentialDistribution(slambda).pdf(sx2),
    sympy.stats.frv_types.BernoulliDistribution(sp1, 1, 0).pmf(1)*sympy.stats.crv_types.NormalDistribution(smu, ssigma).pdf(sx2)]])
M = M1 * M2
E = sprior * sympy.UnevaluatedExpr(M)
E
```

$\displaystyle \frac{e^{- \lambda} e^{- \frac{\mu^{2}}{10000}} e^{- 2 \sigma} \left[\begin{matrix}\lambda^{2} \left(1 - p_{1}\right)^{2} e^{- \lambda x_{1}} e^{- \lambda x_{2}} & \frac{\sqrt{2} \lambda p_{1} \left(1 - p_{1}\right) e^{- \lambda x_{1}} e^{- \frac{\left(- \mu + x_{2}\right)^{2}}{2 \sigma^{2}}}}{2 \sqrt{\pi} \sigma}\\\frac{\sqrt{2} \lambda p_{1} \left(1 - p_{1}\right) e^{- \lambda x_{2}} e^{- \frac{\left(- \mu + x_{1}\right)^{2}}{2 \sigma^{2}}}}{2 \sqrt{\pi} \sigma} & \frac{p_{1}^{2} e^{- \frac{\left(- \mu + x_{1}\right)^{2}}{2 \sigma^{2}}} e^{- \frac{\left(- \mu + x_{2}\right)^{2}}{2 \sigma^{2}}}}{2 \pi \sigma^{2}}\end{matrix}\right]}{20000 \pi}$

```python
E_ = E.doit()
```

We can receive concrete values for a situation like if we assume that both data-points are from the exponential distribution:

```python
E_[0,0]
```

$\displaystyle \frac{\lambda^{2} \left(1 - p_{1}\right)^{2} e^{- \lambda} e^{- \frac{\mu^{2}}{10000}} e^{- 2 \sigma} e^{- \lambda x_{1}} e^{- \lambda x_{2}}}{20000 \pi}$

The marginalization works again by summing over the discrete variables. This time we have to use `sympy.Add` rather than `sympy.summation`, because we don't perform a symbolic summation:

```python
smixture_marginal_2 = sympy.Add(*[E_[i, j] for i in range(2) for j in range(2)])
smixture_marginal_2
```

$\displaystyle \frac{\lambda^{2} \left(1 - p_{1}\right)^{2} e^{- \lambda} e^{- \frac{\mu^{2}}{10000}} e^{- 2 \sigma} e^{- \lambda x_{1}} e^{- \lambda x_{2}}}{20000 \pi} + \frac{\sqrt{2} \lambda p_{1} \left(1 - p_{1}\right) e^{- \lambda} e^{- \frac{\mu^{2}}{10000}} e^{- 2 \sigma} e^{- \lambda x_{2}} e^{- \frac{\left(- \mu + x_{1}\right)^{2}}{2 \sigma^{2}}}}{40000 \pi^{\frac{3}{2}} \sigma} + \frac{\sqrt{2} \lambda p_{1} \left(1 - p_{1}\right) e^{- \lambda} e^{- \frac{\mu^{2}}{10000}} e^{- 2 \sigma} e^{- \lambda x_{1}} e^{- \frac{\left(- \mu + x_{2}\right)^{2}}{2 \sigma^{2}}}}{40000 \pi^{\frac{3}{2}} \sigma} + \frac{p_{1}^{2} e^{- \lambda} e^{- \frac{\mu^{2}}{10000}} e^{- 2 \sigma} e^{- \frac{\left(- \mu + x_{1}\right)^{2}}{2 \sigma^{2}}} e^{- \frac{\left(- \mu + x_{2}\right)^{2}}{2 \sigma^{2}}}}{40000 \pi^{2} \sigma^{2}}$

In this form it is not immediately obvious that the two expressions are actually the same, but we can let SymPy answer that by subtracting the two terms and showing that the result is 0:

```python
sympy.simplify(smixture_marginal_1 - smixture_marginal_2)
```

$\displaystyle 0$


#Install dependencies (Qiskit)

```
# install dependencies (Qiskit)
!pip install --upgrade qiskit==0.19.6
!pip install --upgrade metakernel
!pip install --upgrade portalocker
!pip install --upgrade matplotlib
!pip install git+https://github.com/qiskit-community/qiskit-textbook.git#subdirectory=qiskit-textbook-src
```

    Requirement already up-to-date: qiskit==0.19.6 in /usr/local/lib/python3.6/dist-packages (0.19.6)
    Requirement already satisfied, skipping upgrade: qiskit-ibmq-provider==0.7.2 in /usr/local/lib/python3.6/dist-packages (from qiskit==0.19.6) (0.7.2)
    Requirement already satisfied, skipping upgrade: qiskit-aer==0.5.2 in /usr/local/lib/python3.6/dist-packages (from qiskit==0.19.6) (0.5.2)
    Requirement already satisfied, skipping upgrade: qiskit-terra==0.14.2 in /usr/local/lib/python3.6/dist-packages (from qiskit==0.19.6) (0.14.2)
    Requirement already satisfied, skipping upgrade: qiskit-ignis==0.3.3 in /usr/local/lib/python3.6/dist-packages (from qiskit==0.19.6) (0.3.3)
    Requirement already satisfied, skipping upgrade: qiskit-aqua==0.7.3 in /usr/local/lib/python3.6/dist-packages (from qiskit==0.19.6) (0.7.3)
    Requirement already satisfied, skipping upgrade: requests>=2.19 in /usr/local/lib/python3.6/dist-packages (from qiskit-ibmq-provider==0.7.2->qiskit==0.19.6) (2.23.0)
    Requirement already satisfied, skipping upgrade: numpy>=1.13 in /usr/local/lib/python3.6/dist-packages (from qiskit-ibmq-provider==0.7.2->qiskit==0.19.6) (1.18.5)
    Requirement already satisfied, skipping upgrade: python-dateutil>=2.8.0 in /usr/local/lib/python3.6/dist-packages (from qiskit-ibmq-provider==0.7.2->qiskit==0.19.6) (2.8.1)
    Requirement already satisfied, skipping upgrade: nest-asyncio!=1.1.0,>=1.0.0 in /usr/local/lib/python3.6/dist-packages (from qiskit-ibmq-provider==0.7.2->qiskit==0.19.6) (1.4.0)
    Requirement already satisfied, skipping upgrade: requests-ntlm>=1.1.0 in /usr/local/lib/python3.6/dist-packages (from qiskit-ibmq-provider==0.7.2->qiskit==0.19.6) (1.1.0)
    Requirement already satisfied, skipping upgrade: urllib3>=1.21.1 in /usr/local/lib/python3.6/dist-packages (from qiskit-ibmq-provider==0.7.2->qiskit==0.19.6) (1.24.3)
    Requirement already satisfied, skipping upgrade: websockets<8,>=7 in /usr/local/lib/python3.6/dist-packages (from qiskit-ibmq-provider==0.7.2->qiskit==0.19.6) (7.0)
    Requirement already satisfied, skipping upgrade: pybind11>=2.4 in /usr/local/lib/python3.6/dist-packages (from qiskit-aer==0.5.2->qiskit==0.19.6) (2.5.0)
    Requirement already satisfied, skipping upgrade: cython>=0.27.1 in /usr/local/lib/python3.6/dist-packages (from qiskit-aer==0.5.2->qiskit==0.19.6) (0.29.21)
    Requirement already satisfied, skipping upgrade: scipy>=1.0; python_version > "3.5" in /usr/local/lib/python3.6/dist-packages (from qiskit-aer==0.5.2->qiskit==0.19.6) (1.4.1)
    Requirement already satisfied, skipping upgrade: dill>=0.3 in /usr/local/lib/python3.6/dist-packages (from qiskit-terra==0.14.2->qiskit==0.19.6) (0.3.2)
    Requirement already satisfied, skipping upgrade: fastjsonschema>=2.10 in /usr/local/lib/python3.6/dist-packages (from qiskit-terra==0.14.2->qiskit==0.19.6) (2.14.4)
    Requirement already satisfied, skipping upgrade: marshmallow<4,>=3 in /usr/local/lib/python3.6/dist-packages (from qiskit-terra==0.14.2->qiskit==0.19.6) (3.7.1)
    Requirement already satisfied, skipping upgrade: retworkx>=0.3.2 in /usr/local/lib/python3.6/dist-packages (from qiskit-terra==0.14.2->qiskit==0.19.6) (0.4.0)
    Requirement already satisfied, skipping upgrade: jsonschema>=2.6 in /usr/local/lib/python3.6/dist-packages (from qiskit-terra==0.14.2->qiskit==0.19.6) (2.6.0)
    Requirement already satisfied, skipping upgrade: python-constraint>=1.4 in /usr/local/lib/python3.6/dist-packages (from qiskit-terra==0.14.2->qiskit==0.19.6) (1.4.0)
    Requirement already satisfied, skipping upgrade: networkx>=2.2; python_version > "3.5" in /usr/local/lib/python3.6/dist-packages (from qiskit-terra==0.14.2->qiskit==0.19.6) (2.4)
    Requirement already satisfied, skipping upgrade: marshmallow-polyfield<6,>=5.7 in /usr/local/lib/python3.6/dist-packages (from qiskit-terra==0.14.2->qiskit==0.19.6) (5.9)
    Requirement already satisfied, skipping upgrade: psutil>=5 in /usr/local/lib/python3.6/dist-packages (from qiskit-terra==0.14.2->qiskit==0.19.6) (5.4.8)
    Requirement already satisfied, skipping upgrade: ply>=3.10 in /usr/local/lib/python3.6/dist-packages (from qiskit-terra==0.14.2->qiskit==0.19.6) (3.11)
    Requirement already satisfied, skipping upgrade: sympy>=1.3 in /usr/local/lib/python3.6/dist-packages (from qiskit-terra==0.14.2->qiskit==0.19.6) (1.6.2)
    Requirement already satisfied, skipping upgrade: scikit-learn>=0.17 in /usr/local/lib/python3.6/dist-packages (from qiskit-ignis==0.3.3->qiskit==0.19.6) (0.22.2.post1)
    Requirement already satisfied, skipping upgrade: setuptools>=40.1.0 in /usr/local/lib/python3.6/dist-packages (from qiskit-ignis==0.3.3->qiskit==0.19.6) (49.2.0)
    Requirement already satisfied, skipping upgrade: h5py in /usr/local/lib/python3.6/dist-packages (from qiskit-aqua==0.7.3->qiskit==0.19.6) (2.10.0)
    Requirement already satisfied, skipping upgrade: fastdtw in /usr/local/lib/python3.6/dist-packages (from qiskit-aqua==0.7.3->qiskit==0.19.6) (0.3.4)
    Requirement already satisfied, skipping upgrade: dlx in /usr/local/lib/python3.6/dist-packages (from qiskit-aqua==0.7.3->qiskit==0.19.6) (1.0.4)
    Requirement already satisfied, skipping upgrade: docplex in /usr/local/lib/python3.6/dist-packages (from qiskit-aqua==0.7.3->qiskit==0.19.6) (2.15.194)
    Requirement already satisfied, skipping upgrade: pyscf; sys_platform != "win32" in /usr/local/lib/python3.6/dist-packages (from qiskit-aqua==0.7.3->qiskit==0.19.6) (1.7.4)
    Requirement already satisfied, skipping upgrade: quandl in /usr/local/lib/python3.6/dist-packages (from qiskit-aqua==0.7.3->qiskit==0.19.6) (3.5.2)
    Requirement already satisfied, skipping upgrade: certifi>=2017.4.17 in /usr/local/lib/python3.6/dist-packages (from requests>=2.19->qiskit-ibmq-provider==0.7.2->qiskit==0.19.6) (2020.6.20)
    Requirement already satisfied, skipping upgrade: chardet<4,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from requests>=2.19->qiskit-ibmq-provider==0.7.2->qiskit==0.19.6) (3.0.4)
    Requirement already satisfied, skipping upgrade: idna<3,>=2.5 in /usr/local/lib/python3.6/dist-packages (from requests>=2.19->qiskit-ibmq-provider==0.7.2->qiskit==0.19.6) (2.10)
    Requirement already satisfied, skipping upgrade: six>=1.5 in /usr/local/lib/python3.6/dist-packages (from python-dateutil>=2.8.0->qiskit-ibmq-provider==0.7.2->qiskit==0.19.6) (1.15.0)
    Requirement already satisfied, skipping upgrade: ntlm-auth>=1.0.2 in /usr/local/lib/python3.6/dist-packages (from requests-ntlm>=1.1.0->qiskit-ibmq-provider==0.7.2->qiskit==0.19.6) (1.5.0)
    Requirement already satisfied, skipping upgrade: cryptography>=1.3 in /usr/local/lib/python3.6/dist-packages (from requests-ntlm>=1.1.0->qiskit-ibmq-provider==0.7.2->qiskit==0.19.6) (3.0)
    Requirement already satisfied, skipping upgrade: decorator>=4.3.0 in /usr/local/lib/python3.6/dist-packages (from networkx>=2.2; python_version > "3.5"->qiskit-terra==0.14.2->qiskit==0.19.6) (4.4.2)
    Requirement already satisfied, skipping upgrade: mpmath>=0.19 in /usr/local/lib/python3.6/dist-packages (from sympy>=1.3->qiskit-terra==0.14.2->qiskit==0.19.6) (1.1.0)
    Requirement already satisfied, skipping upgrade: joblib>=0.11 in /usr/local/lib/python3.6/dist-packages (from scikit-learn>=0.17->qiskit-ignis==0.3.3->qiskit==0.19.6) (0.16.0)
    Requirement already satisfied, skipping upgrade: inflection>=0.3.1 in /usr/local/lib/python3.6/dist-packages (from quandl->qiskit-aqua==0.7.3->qiskit==0.19.6) (0.5.0)
    Requirement already satisfied, skipping upgrade: more-itertools in /usr/local/lib/python3.6/dist-packages (from quandl->qiskit-aqua==0.7.3->qiskit==0.19.6) (8.4.0)
    Requirement already satisfied, skipping upgrade: pandas>=0.14 in /usr/local/lib/python3.6/dist-packages (from quandl->qiskit-aqua==0.7.3->qiskit==0.19.6) (1.0.5)
    Requirement already satisfied, skipping upgrade: cffi!=1.11.3,>=1.8 in /usr/local/lib/python3.6/dist-packages (from cryptography>=1.3->requests-ntlm>=1.1.0->qiskit-ibmq-provider==0.7.2->qiskit==0.19.6) (1.14.1)
    Requirement already satisfied, skipping upgrade: pytz>=2017.2 in /usr/local/lib/python3.6/dist-packages (from pandas>=0.14->quandl->qiskit-aqua==0.7.3->qiskit==0.19.6) (2018.9)
    Requirement already satisfied, skipping upgrade: pycparser in /usr/local/lib/python3.6/dist-packages (from cffi!=1.11.3,>=1.8->cryptography>=1.3->requests-ntlm>=1.1.0->qiskit-ibmq-provider==0.7.2->qiskit==0.19.6) (2.20)
    Requirement already up-to-date: metakernel in /usr/local/lib/python3.6/dist-packages (0.24.4)
    Requirement already satisfied, skipping upgrade: pexpect>=4.2 in /usr/local/lib/python3.6/dist-packages (from metakernel) (4.8.0)
    Requirement already satisfied, skipping upgrade: ipykernel in /usr/local/lib/python3.6/dist-packages (from metakernel) (4.10.1)
    Requirement already satisfied, skipping upgrade: ptyprocess>=0.5 in /usr/local/lib/python3.6/dist-packages (from pexpect>=4.2->metakernel) (0.6.0)
    Requirement already satisfied, skipping upgrade: ipython>=4.0.0 in /usr/local/lib/python3.6/dist-packages (from ipykernel->metakernel) (5.5.0)
    Requirement already satisfied, skipping upgrade: jupyter-client in /usr/local/lib/python3.6/dist-packages (from ipykernel->metakernel) (5.3.5)
    Requirement already satisfied, skipping upgrade: tornado>=4.0 in /usr/local/lib/python3.6/dist-packages (from ipykernel->metakernel) (5.1.1)
    Requirement already satisfied, skipping upgrade: traitlets>=4.1.0 in /usr/local/lib/python3.6/dist-packages (from ipykernel->metakernel) (4.3.3)
    Requirement already satisfied, skipping upgrade: prompt-toolkit<2.0.0,>=1.0.4 in /usr/local/lib/python3.6/dist-packages (from ipython>=4.0.0->ipykernel->metakernel) (1.0.18)
    Requirement already satisfied, skipping upgrade: decorator in /usr/local/lib/python3.6/dist-packages (from ipython>=4.0.0->ipykernel->metakernel) (4.4.2)
    Requirement already satisfied, skipping upgrade: setuptools>=18.5 in /usr/local/lib/python3.6/dist-packages (from ipython>=4.0.0->ipykernel->metakernel) (49.2.0)
    Requirement already satisfied, skipping upgrade: pickleshare in /usr/local/lib/python3.6/dist-packages (from ipython>=4.0.0->ipykernel->metakernel) (0.7.5)
    Requirement already satisfied, skipping upgrade: simplegeneric>0.8 in /usr/local/lib/python3.6/dist-packages (from ipython>=4.0.0->ipykernel->metakernel) (0.8.1)
    Requirement already satisfied, skipping upgrade: pygments in /usr/local/lib/python3.6/dist-packages (from ipython>=4.0.0->ipykernel->metakernel) (2.1.3)
    Requirement already satisfied, skipping upgrade: pyzmq>=13 in /usr/local/lib/python3.6/dist-packages (from jupyter-client->ipykernel->metakernel) (19.0.2)
    Requirement already satisfied, skipping upgrade: jupyter-core>=4.6.0 in /usr/local/lib/python3.6/dist-packages (from jupyter-client->ipykernel->metakernel) (4.6.3)
    Requirement already satisfied, skipping upgrade: python-dateutil>=2.1 in /usr/local/lib/python3.6/dist-packages (from jupyter-client->ipykernel->metakernel) (2.8.1)
    Requirement already satisfied, skipping upgrade: ipython-genutils in /usr/local/lib/python3.6/dist-packages (from traitlets>=4.1.0->ipykernel->metakernel) (0.2.0)
    Requirement already satisfied, skipping upgrade: six in /usr/local/lib/python3.6/dist-packages (from traitlets>=4.1.0->ipykernel->metakernel) (1.15.0)
    Requirement already satisfied, skipping upgrade: wcwidth in /usr/local/lib/python3.6/dist-packages (from prompt-toolkit<2.0.0,>=1.0.4->ipython>=4.0.0->ipykernel->metakernel) (0.2.5)
    Requirement already up-to-date: portalocker in /usr/local/lib/python3.6/dist-packages (2.0.0)
    Requirement already up-to-date: matplotlib in /usr/local/lib/python3.6/dist-packages (3.3.1)
    Requirement already satisfied, skipping upgrade: certifi>=2020.06.20 in /usr/local/lib/python3.6/dist-packages (from matplotlib) (2020.6.20)
    Requirement already satisfied, skipping upgrade: pillow>=6.2.0 in /usr/local/lib/python3.6/dist-packages (from matplotlib) (7.0.0)
    Requirement already satisfied, skipping upgrade: cycler>=0.10 in /usr/local/lib/python3.6/dist-packages (from matplotlib) (0.10.0)
    Requirement already satisfied, skipping upgrade: python-dateutil>=2.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib) (2.8.1)
    Requirement already satisfied, skipping upgrade: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.3 in /usr/local/lib/python3.6/dist-packages (from matplotlib) (2.4.7)
    Requirement already satisfied, skipping upgrade: numpy>=1.15 in /usr/local/lib/python3.6/dist-packages (from matplotlib) (1.18.5)
    Requirement already satisfied, skipping upgrade: kiwisolver>=1.0.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib) (1.2.0)
    Requirement already satisfied, skipping upgrade: six in /usr/local/lib/python3.6/dist-packages (from cycler>=0.10->matplotlib) (1.15.0)
    Collecting git+https://github.com/qiskit-community/qiskit-textbook.git#subdirectory=qiskit-textbook-src
      Cloning https://github.com/qiskit-community/qiskit-textbook.git to /tmp/pip-req-build-v11l3b5p
      Running command git clone -q https://github.com/qiskit-community/qiskit-textbook.git /tmp/pip-req-build-v11l3b5p
    Requirement already satisfied: qiskit in /usr/local/lib/python3.6/dist-packages (from qiskit-textbook==0.1.0) (0.19.6)
    Requirement already satisfied: ipython in /usr/local/lib/python3.6/dist-packages (from qiskit-textbook==0.1.0) (5.5.0)
    Requirement already satisfied: ipywidgets in /usr/local/lib/python3.6/dist-packages (from qiskit-textbook==0.1.0) (7.5.1)
    Requirement already satisfied: numpy in /usr/local/lib/python3.6/dist-packages (from qiskit-textbook==0.1.0) (1.18.5)
    Requirement already satisfied: matplotlib in /usr/local/lib/python3.6/dist-packages (from qiskit-textbook==0.1.0) (3.3.1)
    Requirement already satisfied: qiskit-aer==0.5.2 in /usr/local/lib/python3.6/dist-packages (from qiskit->qiskit-textbook==0.1.0) (0.5.2)
    Requirement already satisfied: qiskit-terra==0.14.2 in /usr/local/lib/python3.6/dist-packages (from qiskit->qiskit-textbook==0.1.0) (0.14.2)
    Requirement already satisfied: qiskit-aqua==0.7.3 in /usr/local/lib/python3.6/dist-packages (from qiskit->qiskit-textbook==0.1.0) (0.7.3)
    Requirement already satisfied: qiskit-ignis==0.3.3 in /usr/local/lib/python3.6/dist-packages (from qiskit->qiskit-textbook==0.1.0) (0.3.3)
    Requirement already satisfied: qiskit-ibmq-provider==0.7.2 in /usr/local/lib/python3.6/dist-packages (from qiskit->qiskit-textbook==0.1.0) (0.7.2)
    Requirement already satisfied: pickleshare in /usr/local/lib/python3.6/dist-packages (from ipython->qiskit-textbook==0.1.0) (0.7.5)
    Requirement already satisfied: simplegeneric>0.8 in /usr/local/lib/python3.6/dist-packages (from ipython->qiskit-textbook==0.1.0) (0.8.1)
    Requirement already satisfied: traitlets>=4.2 in /usr/local/lib/python3.6/dist-packages (from ipython->qiskit-textbook==0.1.0) (4.3.3)
    Requirement already satisfied: prompt-toolkit<2.0.0,>=1.0.4 in /usr/local/lib/python3.6/dist-packages (from ipython->qiskit-textbook==0.1.0) (1.0.18)
    Requirement already satisfied: pexpect; sys_platform != "win32" in /usr/local/lib/python3.6/dist-packages (from ipython->qiskit-textbook==0.1.0) (4.8.0)
    Requirement already satisfied: pygments in /usr/local/lib/python3.6/dist-packages (from ipython->qiskit-textbook==0.1.0) (2.1.3)
    Requirement already satisfied: setuptools>=18.5 in /usr/local/lib/python3.6/dist-packages (from ipython->qiskit-textbook==0.1.0) (49.2.0)
    Requirement already satisfied: decorator in /usr/local/lib/python3.6/dist-packages (from ipython->qiskit-textbook==0.1.0) (4.4.2)
    Requirement already satisfied: widgetsnbextension~=3.5.0 in /usr/local/lib/python3.6/dist-packages (from ipywidgets->qiskit-textbook==0.1.0) (3.5.1)
    Requirement already satisfied: ipykernel>=4.5.1 in /usr/local/lib/python3.6/dist-packages (from ipywidgets->qiskit-textbook==0.1.0) (4.10.1)
    Requirement already satisfied: nbformat>=4.2.0 in /usr/local/lib/python3.6/dist-packages (from ipywidgets->qiskit-textbook==0.1.0) (5.0.7)
    Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib->qiskit-textbook==0.1.0) (2.8.1)
    Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib->qiskit-textbook==0.1.0) (1.2.0)
    Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.3 in /usr/local/lib/python3.6/dist-packages (from matplotlib->qiskit-textbook==0.1.0) (2.4.7)
    Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.6/dist-packages (from matplotlib->qiskit-textbook==0.1.0) (0.10.0)
    Requirement already satisfied: pillow>=6.2.0 in /usr/local/lib/python3.6/dist-packages (from matplotlib->qiskit-textbook==0.1.0) (7.0.0)
    Requirement already satisfied: certifi>=2020.06.20 in /usr/local/lib/python3.6/dist-packages (from matplotlib->qiskit-textbook==0.1.0) (2020.6.20)
    Requirement already satisfied: pybind11>=2.4 in /usr/local/lib/python3.6/dist-packages (from qiskit-aer==0.5.2->qiskit->qiskit-textbook==0.1.0) (2.5.0)
    Requirement already satisfied: scipy>=1.0; python_version > "3.5" in /usr/local/lib/python3.6/dist-packages (from qiskit-aer==0.5.2->qiskit->qiskit-textbook==0.1.0) (1.4.1)
    Requirement already satisfied: cython>=0.27.1 in /usr/local/lib/python3.6/dist-packages (from qiskit-aer==0.5.2->qiskit->qiskit-textbook==0.1.0) (0.29.21)
    Requirement already satisfied: sympy>=1.3 in /usr/local/lib/python3.6/dist-packages (from qiskit-terra==0.14.2->qiskit->qiskit-textbook==0.1.0) (1.6.2)
    Requirement already satisfied: dill>=0.3 in /usr/local/lib/python3.6/dist-packages (from qiskit-terra==0.14.2->qiskit->qiskit-textbook==0.1.0) (0.3.2)
    Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.6/dist-packages (from qiskit-terra==0.14.2->qiskit->qiskit-textbook==0.1.0) (2.6.0)
    Requirement already satisfied: python-constraint>=1.4 in /usr/local/lib/python3.6/dist-packages (from qiskit-terra==0.14.2->qiskit->qiskit-textbook==0.1.0) (1.4.0)
    Requirement already satisfied: psutil>=5 in /usr/local/lib/python3.6/dist-packages (from qiskit-terra==0.14.2->qiskit->qiskit-textbook==0.1.0) (5.4.8)
    Requirement already satisfied: fastjsonschema>=2.10 in /usr/local/lib/python3.6/dist-packages (from qiskit-terra==0.14.2->qiskit->qiskit-textbook==0.1.0) (2.14.4)
    Requirement already satisfied: retworkx>=0.3.2 in /usr/local/lib/python3.6/dist-packages (from qiskit-terra==0.14.2->qiskit->qiskit-textbook==0.1.0) (0.4.0)
    Requirement already satisfied: networkx>=2.2; python_version > "3.5" in /usr/local/lib/python3.6/dist-packages (from qiskit-terra==0.14.2->qiskit->qiskit-textbook==0.1.0) (2.4)
    Requirement already satisfied: marshmallow-polyfield<6,>=5.7 in /usr/local/lib/python3.6/dist-packages (from qiskit-terra==0.14.2->qiskit->qiskit-textbook==0.1.0) (5.9)
    Requirement already satisfied: ply>=3.10 in /usr/local/lib/python3.6/dist-packages (from qiskit-terra==0.14.2->qiskit->qiskit-textbook==0.1.0) (3.11)
    Requirement already satisfied: marshmallow<4,>=3 in /usr/local/lib/python3.6/dist-packages (from qiskit-terra==0.14.2->qiskit->qiskit-textbook==0.1.0) (3.7.1)
    Requirement already satisfied: dlx in /usr/local/lib/python3.6/dist-packages (from qiskit-aqua==0.7.3->qiskit->qiskit-textbook==0.1.0) (1.0.4)
    Requirement already satisfied: fastdtw in /usr/local/lib/python3.6/dist-packages (from qiskit-aqua==0.7.3->qiskit->qiskit-textbook==0.1.0) (0.3.4)
    Requirement already satisfied: pyscf; sys_platform != "win32" in /usr/local/lib/python3.6/dist-packages (from qiskit-aqua==0.7.3->qiskit->qiskit-textbook==0.1.0) (1.7.4)
    Requirement already satisfied: h5py in /usr/local/lib/python3.6/dist-packages (from qiskit-aqua==0.7.3->qiskit->qiskit-textbook==0.1.0) (2.10.0)
    Requirement already satisfied: scikit-learn>=0.20.0 in /usr/local/lib/python3.6/dist-packages (from qiskit-aqua==0.7.3->qiskit->qiskit-textbook==0.1.0) (0.22.2.post1)
    Requirement already satisfied: quandl in /usr/local/lib/python3.6/dist-packages (from qiskit-aqua==0.7.3->qiskit->qiskit-textbook==0.1.0) (3.5.2)
    Requirement already satisfied: docplex in /usr/local/lib/python3.6/dist-packages (from qiskit-aqua==0.7.3->qiskit->qiskit-textbook==0.1.0) (2.15.194)
    Requirement already satisfied: websockets<8,>=7 in /usr/local/lib/python3.6/dist-packages (from qiskit-ibmq-provider==0.7.2->qiskit->qiskit-textbook==0.1.0) (7.0)
    Requirement already satisfied: urllib3>=1.21.1 in /usr/local/lib/python3.6/dist-packages (from qiskit-ibmq-provider==0.7.2->qiskit->qiskit-textbook==0.1.0) (1.24.3)
    Requirement already satisfied: requests>=2.19 in /usr/local/lib/python3.6/dist-packages (from qiskit-ibmq-provider==0.7.2->qiskit->qiskit-textbook==0.1.0) (2.23.0)
    Requirement already satisfied: requests-ntlm>=1.1.0 in /usr/local/lib/python3.6/dist-packages (from qiskit-ibmq-provider==0.7.2->qiskit->qiskit-textbook==0.1.0) (1.1.0)
    Requirement already satisfied: nest-asyncio!=1.1.0,>=1.0.0 in /usr/local/lib/python3.6/dist-packages (from qiskit-ibmq-provider==0.7.2->qiskit->qiskit-textbook==0.1.0) (1.4.0)
    Requirement already satisfied: ipython-genutils in /usr/local/lib/python3.6/dist-packages (from traitlets>=4.2->ipython->qiskit-textbook==0.1.0) (0.2.0)
    Requirement already satisfied: six in /usr/local/lib/python3.6/dist-packages (from traitlets>=4.2->ipython->qiskit-textbook==0.1.0) (1.15.0)
    Requirement already satisfied: wcwidth in /usr/local/lib/python3.6/dist-packages (from prompt-toolkit<2.0.0,>=1.0.4->ipython->qiskit-textbook==0.1.0) (0.2.5)
    Requirement already satisfied: ptyprocess>=0.5 in /usr/local/lib/python3.6/dist-packages (from pexpect; sys_platform != "win32"->ipython->qiskit-textbook==0.1.0) (0.6.0)
    Requirement already satisfied: notebook>=4.4.1 in /usr/local/lib/python3.6/dist-packages (from widgetsnbextension~=3.5.0->ipywidgets->qiskit-textbook==0.1.0) (5.3.1)
    Requirement already satisfied: tornado>=4.0 in /usr/local/lib/python3.6/dist-packages (from ipykernel>=4.5.1->ipywidgets->qiskit-textbook==0.1.0) (5.1.1)
    Requirement already satisfied: jupyter-client in /usr/local/lib/python3.6/dist-packages (from ipykernel>=4.5.1->ipywidgets->qiskit-textbook==0.1.0) (5.3.5)
    Requirement already satisfied: jupyter-core in /usr/local/lib/python3.6/dist-packages (from nbformat>=4.2.0->ipywidgets->qiskit-textbook==0.1.0) (4.6.3)
    Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.6/dist-packages (from sympy>=1.3->qiskit-terra==0.14.2->qiskit->qiskit-textbook==0.1.0) (1.1.0)
    Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.6/dist-packages (from scikit-learn>=0.20.0->qiskit-aqua==0.7.3->qiskit->qiskit-textbook==0.1.0) (0.16.0)
    Requirement already satisfied: inflection>=0.3.1 in /usr/local/lib/python3.6/dist-packages (from quandl->qiskit-aqua==0.7.3->qiskit->qiskit-textbook==0.1.0) (0.5.0)
    Requirement already satisfied: pandas>=0.14 in /usr/local/lib/python3.6/dist-packages (from quandl->qiskit-aqua==0.7.3->qiskit->qiskit-textbook==0.1.0) (1.0.5)
    Requirement already satisfied: more-itertools in /usr/local/lib/python3.6/dist-packages (from quandl->qiskit-aqua==0.7.3->qiskit->qiskit-textbook==0.1.0) (8.4.0)
    Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.6/dist-packages (from requests>=2.19->qiskit-ibmq-provider==0.7.2->qiskit->qiskit-textbook==0.1.0) (2.10)
    Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from requests>=2.19->qiskit-ibmq-provider==0.7.2->qiskit->qiskit-textbook==0.1.0) (3.0.4)
    Requirement already satisfied: ntlm-auth>=1.0.2 in /usr/local/lib/python3.6/dist-packages (from requests-ntlm>=1.1.0->qiskit-ibmq-provider==0.7.2->qiskit->qiskit-textbook==0.1.0) (1.5.0)
    Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python3.6/dist-packages (from requests-ntlm>=1.1.0->qiskit-ibmq-provider==0.7.2->qiskit->qiskit-textbook==0.1.0) (3.0)
    Requirement already satisfied: nbconvert in /usr/local/lib/python3.6/dist-packages (from notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets->qiskit-textbook==0.1.0) (5.6.1)
    Requirement already satisfied: Send2Trash in /usr/local/lib/python3.6/dist-packages (from notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets->qiskit-textbook==0.1.0) (1.5.0)
    Requirement already satisfied: terminado>=0.8.1 in /usr/local/lib/python3.6/dist-packages (from notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets->qiskit-textbook==0.1.0) (0.8.3)
    Requirement already satisfied: jinja2 in /usr/local/lib/python3.6/dist-packages (from notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets->qiskit-textbook==0.1.0) (2.11.2)
    Requirement already satisfied: pyzmq>=13 in /usr/local/lib/python3.6/dist-packages (from jupyter-client->ipykernel>=4.5.1->ipywidgets->qiskit-textbook==0.1.0) (19.0.2)
    Requirement already satisfied: pytz>=2017.2 in /usr/local/lib/python3.6/dist-packages (from pandas>=0.14->quandl->qiskit-aqua==0.7.3->qiskit->qiskit-textbook==0.1.0) (2018.9)
    Requirement already satisfied: cffi!=1.11.3,>=1.8 in /usr/local/lib/python3.6/dist-packages (from cryptography>=1.3->requests-ntlm>=1.1.0->qiskit-ibmq-provider==0.7.2->qiskit->qiskit-textbook==0.1.0) (1.14.1)
    Requirement already satisfied: defusedxml in /usr/local/lib/python3.6/dist-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets->qiskit-textbook==0.1.0) (0.6.0)
    Requirement already satisfied: mistune<2,>=0.8.1 in /usr/local/lib/python3.6/dist-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets->qiskit-textbook==0.1.0) (0.8.4)
    Requirement already satisfied: entrypoints>=0.2.2 in /usr/local/lib/python3.6/dist-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets->qiskit-textbook==0.1.0) (0.3)
    Requirement already satisfied: bleach in /usr/local/lib/python3.6/dist-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets->qiskit-textbook==0.1.0) (3.1.5)
    Requirement already satisfied: pandocfilters>=1.4.1 in /usr/local/lib/python3.6/dist-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets->qiskit-textbook==0.1.0) (1.4.2)
    Requirement already satisfied: testpath in /usr/local/lib/python3.6/dist-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets->qiskit-textbook==0.1.0) (0.4.4)
    Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python3.6/dist-packages (from jinja2->notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets->qiskit-textbook==0.1.0) (1.1.1)
    Requirement already satisfied: pycparser in /usr/local/lib/python3.6/dist-packages (from cffi!=1.11.3,>=1.8->cryptography>=1.3->requests-ntlm>=1.1.0->qiskit-ibmq-provider==0.7.2->qiskit->qiskit-textbook==0.1.0) (2.20)
    Requirement already satisfied: webencodings in /usr/local/lib/python3.6/dist-packages (from bleach->nbconvert->notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets->qiskit-textbook==0.1.0) (0.5.1)
    Requirement already satisfied: packaging in /usr/local/lib/python3.6/dist-packages (from bleach->nbconvert->notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets->qiskit-textbook==0.1.0) (20.4)
    Building wheels for collected packages: qiskit-textbook
      Building wheel for qiskit-textbook (setup.py) ... done
      Created wheel for qiskit-textbook: filename=qiskit_textbook-0.1.0-cp36-none-any.whl size=16672 sha256=91226ef6dcb84513a75902c1b5c02760c5a1e7d33ec74aa39fbd1e03042fc31f
      Stored in directory: /tmp/pip-ephem-wheel-cache-o72zdjj6/wheels/0c/57/26/1c12886a219af04a38ca1e674fdeeff40b8a5963326ba3c71a
    Successfully built qiskit-textbook
    Installing collected packages: qiskit-textbook
    Successfully installed qiskit-textbook-0.1.0

#Import Basic Quantum Circuit
#引入基本的量子電路

# Single Quantum Gate - XGate
# 單量子閘-X閘

```
#XGate

#Importing QuantumCircuit from qikit
from qiskit import QuantumCircuit

#Creating a quantum cicuit with 1 Qubit
qc = QuantumCircuit(1)

#Creating XGate for q0
qc.x(0)

#Drawing the quantum circuit
qc.draw('mpl')
```

# Single Quantum Gate - HGate
# 單量子閘-H閘

```
#Importing QuantumCircuit from qikit
from qiskit import QuantumCircuit

#Creating a quantum cicuit with 1 Qubit
qc = QuantumCircuit(1)

#Creating XGate for q0
qc.h(0)

#Drawing the quantum circuit
qc.draw('mpl')
```

#Combining Two Quantum Gates in 1 Qubit
#以1 Qubit組合兩個量子閘

```
#Importing QuantumCircuit from qikit
from qiskit import QuantumCircuit

#Creating a quantum cicuit with 1 Qubit
qc = QuantumCircuit(1)

#Creating XGate for q0
qc.x(0)
qc.h(0)

#Drawing the quantum circuit
qc.draw('mpl')
```

<pre style="word-wrap: normal;white-space: pre;background: #fff0;line-height: 1.1;font-family: "Courier New",Courier,monospace">     ┌───┐┌───┐
q_0: ┤ X ├┤ H ├
     └───┘└───┘</pre>

# Two or more Quantum Gates - Bell State
＃兩個或更多量子閘-貝爾狀態
＃HGate + CNOTx
# H閘 + 受控反閘

```
#Importing QuantumCircuit from qikit
from qiskit import QuantumCircuit

#Creating a quantum cicuit with 2 Qubits and 2 Classical Bits
qc = QuantumCircuit(2,2)

#Creating HGate for q0
qc.h(0)

#Creating CNOTx Gate (Control:q0 and Target: q1)
qc.cx(0,1)

#Drawing the quantum circuit
qc.draw()
```

<pre style="word-wrap: normal;white-space: pre;background: #fff0;line-height: 1.1;font-family: "Courier New",Courier,monospace">     ┌───┐
q_0: ┤ H ├──■──
     └───┘┌─┴─┐
q_1: ─────┤ X ├
          └───┘
c_0: ══════════

c_1: ══════════
               </pre>

# XGate Simulator with default initial state, |0>
# 具有|0>初始狀態的X閘模擬器

```
口口#XGate
from qiskit import QuantumCircuit, Aer, execute
#Nice Latex Arrary
from qiskit_textbook.tools import array_to_latex

#Creating a quantum cicuit with 1 Qubit
qc = QuantumCircuit(1)

#Creating XGate for q0
qc.x(0)

# Let's see the result:
backend = Aer.get_backend('statevector_simulator')
final_state = execute(qc,backend).result().get_statevector()
# Print the statevector neatly:
array_to_latex(final_state, pretext="\\text{Statevector = }")
```

$$
\text{Statevector = }\begin{bmatrix}
0 \\
1
\end{bmatrix}
$$

# XGate Simulator with initial state, |1>
# 具有|1>初始狀態的X閘模擬器

```
#XGate
from qiskit import QuantumCircuit, Aer, execute
#Nice Latex Arrary
from qiskit_textbook.tools import array_to_latex

#Creating a quantum cicuit with 1 Qubit
qc = QuantumCircuit(1)

# Define initial_state as |1>
initial_state = [0,1]

# Apply initialisation operation to the 0th qubit
qc.initialize(initial_state, 0)

#Creating XGate for q0
qc.x(0)

# Let's see the result:
backend = Aer.get_backend('statevector_simulator')
final_state = execute(qc,backend).result().get_statevector()
# Print the statevector neatly:
array_to_latex(final_state, pretext="\\text{Statevector = }")
```

$$
\text{Statevector = }\begin{bmatrix}
1 \\
0
\end{bmatrix}
$$


# Circuit

```python
from discopy import grammar
from pytket.qasm import circuit_to_qasm_str
from pytket.circuit.display import render_circuit_jupyter

from lambeq.ccg2discocat import DepCCGParser
from lambeq.circuit import IQPAnsatz
from lambeq.core.types import AtomicType

N = AtomicType.NOUN
S = AtomicType.SENTENCE
```

```python
depccg_parser = DepCCGParser()
diagram = depccg_parser.sentence2diagram('Alice runs')
diagram.draw()
```

```python
ansatz = IQPAnsatz({N: 1, S: 1}, n_layers=2)
discopy_circuit = ansatz(diagram)
discopy_circuit.draw(figsize=(10, 15))
```

```python
tket_circuit = ansatz(diagram).to_tk()

render_circuit_jupyter(tket_circuit)
```

<!DOCTYPE html>

    <meta charset="UTF-8">

q[0]

q[1]

q[2]

c[0]

c[1]

                    Rx(2*Alice__n_0)

                    H

                    H

                    Rz(2*Alice__n_1)

                    Rz(2*runs__n.r@s_0)

                    Rx(2*Alice__n_2)

                    H

                    H

                    Rz(2*runs__n.r@s_1)

                    H

q[0]

q[1]

q[2]

c[0]

c[1]

```python
from sympy import default_sort_key

# Make sure you sort your symbols as they are returned as a set.
parameters = sorted(tket_circuit.free_symbols(), key=default_sort_key)

param_dict = {p: i * 0.001 for i, p in enumerate(parameters)}
param_dict
```

    {Alice__n_0: 0.0,
     Alice__n_1: 0.001,
     Alice__n_2: 0.002,
     runs__n.r@s_0: 0.003,
     runs__n.r@s_1: 0.004}

```python
tket_circuit.symbol_substitution(param_dict)
render_circuit_jupyter(tket_circuit)
```

<!DOCTYPE html>

    <meta charset="UTF-8">

q[0]

q[1]

q[2]

c[0]

c[1]

                    Rx(0.0)

                    H

                    H

                    Rz(0.002)

                    Rz(0.006)

                    Rx(0.004)

                    H

                    H

                    Rz(0.008)

                    H

q[0]

q[1]

q[2]

c[0]

c[1]


```python
# This cell is mandatory in all Dymos documentation notebooks.
missing_packages = []
try:
    import openmdao.api as om
except ImportError:
    if 'google.colab' in str(get_ipython()):
        !python -m pip install openmdao[notebooks]
    else:
        missing_packages.append('openmdao')
try:
    import dymos as dm
except ImportError:
    if 'google.colab' in str(get_ipython()):
        !python -m pip install dymos
    else:
        missing_packages.append('dymos')
try:
    import pyoptsparse
except ImportError:
    if 'google.colab' in str(get_ipython()):
        !pip install -q condacolab
        import condacolab
        condacolab.install_miniconda()
        !conda install -c conda-forge pyoptsparse
    else:
        missing_packages.append('pyoptsparse')
if missing_packages:
    raise EnvironmentError('This notebook requires the following packages '
                           'please install them and restart this notebook\'s runtime: {",".join(missing_packages)}')
```

# Constraints

Now that we've shown how to add degrees of freedom to a system with variables in the form of
time, states, and controls, we need to look at how to constrain the system.  In optimal control,
constraints typically come in two flavors:  boundary constraints and path constraints.

As OpenMDAO components, outputs of Dymos Phases can be constrained using OpenMDAO's `add_constraint` method, but Dymos Phases provide their own methods to make defining these constraints somewhat simpler.

##  Boundary Constraints

Boundary constraints are constraints on a variable value at the start or end of a phase.  There
are a few different ways to impose these in Dymos, each with slightly different behavior.

Let's consider that we want to solve for the elevation angle that results in the maximum possible
range flown by a cannonball.  In this situation we have some set of initial conditions that are
fixed.

\begin{align}
    t_0 &= 0 \, \mathrm{s} \\
    x_0 &= 0 \, \mathrm{m} \\
    y_0 &= 0 \, \mathrm{m} \\
    v_0 &= 100 \, \frac{\mathrm{m}}{\mathrm{s}}
\end{align}

The first, most obvious way to constrain fixed values is to remove them from the optimization problem altogether.

For time, this is done using the `fix_initial` or `fix_duration` arguments to `set_time_options`.
This also allows `t_initial` and `t_duration` to be provided from an external source via connection, if so desired.

For states and controls, the situation is slightly different.
Rather than providing initial and final values, similar to the way time is handled, the implicit simulation techniques must
be provided state values at the state discretization nodes and control values at *all* nodes.
Instead, for states and controls, the user specifies `fix_initial=True` or `fix_final=True`.

Removing constrained values from the optimization has the following pros and cons.
On the pro side, we're making the optimization problem smaller by omitting them.
On the con side, the optimizer has absolutely no freedom to move these values around even a little.
This can sometimes lead to failure modes that aren't necessarily obvious, especially to new users.

The following example solves the brachistochrone problem by omitting the initial time and initial state, as well as the final position state from the optimization.

The second method for bounding initial/final time, states, or controls is to leave them in the
optimization problem but to constrain only their initial or final values.  For time, this is
accomplished with the options `initial_bounds` and `duration_bounds`.  Each of these takes a tuple
of `(lower, upper)` values that the optimizer must obey when providing new variable values.  Note
that since states and controls may be vector valued, lower and upper may themselves be iterable.
To *pin* the value of a state, time, or control to a value just set lower and upper to the same
value.

As for the pros and cons of this technique, its largely similar to that for the first technique,
but it's somewhat optimizer dependent.  Some optimizers *may* allow bounds on design variables to
be violated slightly (to some small tolerance).  In theory this could alleviate some of the issues
with omitting a design variable altogether, but in practice that's unlikely.

The first two options work by imposing bounds (or by not providing a variable to the optimizer
altogether).  The third option is to pose bound constraints as actual constraints on the NLP.
This is accomplished with the `add_boundary_constraint` method on Phases.

The downside of this technique is that it makes the NLP problem larger, though not by much.  On
the plus side, this method allows the user to constrain any output within the ODE.  If the user
needs to constrain an auxiliary output, this is the only option.  It may also behave somewhat better
in certain circumstances.  Depending on scaling, the NLP may ensure that collocation defects are
satisfied before forcing an infeasible boundary constraint to be satisfied, for instance.

In conclusion, while using `fix_initial=True` for problems with fixed initial conditions is not a bad solution, the generality of `add_boundary_constraint`, especially for terminal constraints that risk being over-constrained, makes it a good first-choice in those situations.
In forward-shooting phases (`solve_segments='forward'`) only the initial values of the states are design variables for the optimizer.
As such, simple bounds on final state values are not possible in those situations, and `add_boundary_constraint` must be used instead.

##  Path Constraints

The second class of constraints supported by Dymos are *path* constraints, so called because they are imposed throughout the entire phase.
Like bound constraints, path constraints can be imposed on design variables using simple bounds.
This is accomplished using the `lower` and `upper` arguments to `add_state`, `add_control`, and `add_parameter`.
(Since time is monotonically increasing or decreasing the notion of a path constraint is irrelevant for it).

For vector-valued states and controls, lower/upper should be dimensioned the same as state or control.
If given as a scalar, it will be applied to all values in the state or control.

```{Note}
Bounds on states in Gauss-Lobatto Phases are **not** equivalent to path constraints.
The values of states in Gauss-Lobatto phases are provided at only the state-transcription nodes and then interpolated to the collocation nodes.
Therefore, the bounds will have no impact on these interpolated values which therefore may not satisfy the bounds, as one might expect.
```

Phases also support the `add_path_constraint` method, which imposes path constraints as constraints in the NLP problem.
As with `add_bound_constraint`, the `add_path_constraint` method is the only option for path constraining an output of the ODE.

The downside of path constraints is that they add a considerable number of constraints to the NLP problem and thus may negatively impact performance, although this is generally minor for many problems.


```python
from IPython.display import Image
Image('../../../python_for_probability_statistics_and_machine_learning.jpg')
```

[Python for Probability, Statistics, and Machine Learning](https://www.springer.com/fr/book/9783319307152)

# Conditional Expectation as Projection

Now that we understand projection methods geometrically, we can apply
them to conditional probability. This is the *key* concept that ties
probability to geometry, optimization, and linear algebra.

### Inner Product for Random Variables

 From our previous work on projection for vectors in
$\mathbb{R}^n$, we have a good geometric grasp on how projection is related to
Minimum Mean Squared Error (MMSE). By one abstract step, we can carry
all of our geometric interpretations to the space of random variables.
For example, we previously noted that at the point of projection, we had the
following orthogonal (i.e.,  perpendicular vectors) condition,

$$
( \mathbf{y} - \mathbf{v}_{opt} )^T \mathbf{v} = 0
$$

 which by noting the inner product slightly more abstractly as
$\langle\mathbf{x},\mathbf{y} \rangle = \mathbf{x}^T \mathbf{y}$, we can
express as

$$
\langle \mathbf{y} - \mathbf{v}_{opt},\mathbf{v} \rangle = 0
$$

 and by defining the inner product for the random variables
$X$ and $Y$ as

$$
\langle X,Y \rangle = \mathbb{E}(X Y)
$$

 we have the same relationship:

$$
\langle X-h_{opt}(Y),Y \rangle = 0
$$

 which holds not for vectors in $\mathbb{R}^n$, but for random
variables $X$ and $Y$ and functions of those random variables. Exactly why this
is true is technical, but it turns out that one can build up the *entire theory
of probability* this way [[edward1987radically]](#edward1987radically), by using the expectation as
an inner product.

Furthermore, by abstracting out the inner product concept, we have connected
minimum-mean-squared-error (MMSE) optimization problems, geometry, and random
variables.  That's  a lot of mileage to get a out of an abstraction and it
enables us to shift between these interpretations to address real problems.
Soon, we'll do this with some examples, but first we collect the most important
result that flows naturally from this abstraction.

### Conditional Expectation as Projection

The conditional expectation is the minimum mean squared error (MMSE) solution
to the following problem [^proof]:

$$
\min_h \int_{\mathbb{R}} (x - h(y) )^2 dx
$$

 with the minimizing $h_{opt}(Y) $ as

$$
h_{opt}(Y) = \mathbb{E}(X|Y)
$$

[^proof]: See appendix for proof using the Cauchy-Schwarz inequality.

 which is another way of saying that among all possible functions
$h(Y)$, the one that minimizes the MSE is $ \mathbb{E}(X|Y)$. From our previous discussion on projection, we noted that
these MMSE solutions can be thought of as projections onto a subspace that
characterizes $Y$. For example, we previously noted that at the point of
projection, we have perpendicular terms,

$$
\begin{equation}
\langle X-h_{opt}(Y),Y \rangle = 0
\end{equation}
\label{eq:ortho} \tag{1}
$$

 but since we know that the MMSE solution

$$
h_{opt}(Y) = \mathbb{E}(X|Y)
$$

 we have by direct substitution,

$$
\begin{equation}
\mathbb{E}(X-\mathbb{E}(X|Y),Y) = 0
\end{equation}
\label{eq:ortho_001} \tag{2}
$$

 That last step seems pretty innocuous, but it ties MMSE to
conditional expectation to the inner project abstraction, and in so doing,
reveals the conditional expectation to be a projection operator for random
variables. Before we develop this further, let's grab some quick dividends.
From the previous equation, by linearity of the expectation, we obtain,

$$
\mathbb{E}(X Y) =  \mathbb{E}(Y \mathbb{E}(X|Y))
$$

 which is the so-called *tower property* of the expectation. Note that
we could have found this by using the formal definition of conditional
expectation,

$$
\mathbb{E}(X|Y) = \int_{\mathbb{R}^2} x \frac{f_{X,Y}(x,y)}{f_Y(y)} dx dy
$$

 and brute-force direct integration,

$$
\mathbb{E}(Y \mathbb{E}(X|Y)) = \int_{\mathbb{R}} y \int_{\mathbb{R}} x \frac{f_{X,Y}(x,y)}{f_Y(y)}  f_Y(y) dx dy
$$

$$
\
                              =\int_{\mathbb{R}^2} x y f_{X,Y}(x,y) dx dy
$$

$$
\
                              =\mathbb{E}( X Y)
$$

 which is not very geometrically intuitive. This lack of geometric
intuition makes it hard to apply these concepts and keep track of these
relationships.

We can keep pursuing this analogy and obtain the length of the error term
from the orthogonality property of the MMSE solution as,

$$
\langle X-h_{opt}(Y),X-h_{opt}(Y)\rangle = \langle X,X  \rangle - \langle h_{opt}(Y),h_{opt}(Y)  \rangle
$$

 and then by substituting all the notation we obtain

$$
\mathbb{E}(X-  \mathbb{E}(X|Y))^2 = \mathbb{E}(X)^2 - \mathbb{E}(\mathbb{E}(X|Y) )^2
$$

 which would be tough to compute by direct integration.

To formally establish that $\mathbb{E}(X|Y)$ *is* in fact *a projection operator* we
need to show idempotency.  Recall that idempotency means that once we project
something onto a subspace, further projections do nothing. In the space of
random variables, $\mathbb{E}(X|\cdot$) is the idempotent projection as we can
show by noting that

$$
h_{opt} = \mathbb{E}(X|Y)
$$

 is purely a function of $Y$, so that

$$
\mathbb{E}(h_{opt}(Y)|Y) = h_{opt}(Y)
$$

 because $Y$ is fixed, this verifies idempotency. Thus, conditional
expectation is the corresponding projection operator for random variables. We
can continue to carry over our geometric interpretations of projections for
vectors ($\mathbf{v}$) into random variables ($X$).  With this important
result, let's consider some examples of conditional expectations obtained by
using brute force to find the optimal MMSE function $h_{opt}$ as well as by
using our new perspective on conditional expectation.

**Example.** Suppose we have a random variable, $X$, then what constant is closest to $X$ in
the sense of the mean-squared-error (MSE)? In other words, which $c \in
\mathbb{R}$ minimizes the following mean squared error:

$$
\mbox{MSE} = \mathbb{E}( X - c )^2
$$

 we can work this out many ways. First, using calculus-based optimization,

$$
\mathbb{E}(X-c)^2=\mathbb{E}(c^2-2 c X + X^2)=c^2-2 c \mathbb{E}(X) + \mathbb{E}(X^2)
$$

 and then take the first derivative with respect to $c$ and solve:

$$
c_{opt}=\mathbb{E}(X)
$$

 Remember that $X$ may potentially take on many values, but this says
that the closest number to $X$ in the MSE sense is $\mathbb{E}(X)$.  This is
intuitively pleasing.  Coming at this same problem using our inner product,
from Equation ref{eq:ortho_001} we know that at the point of projection

$$
\mathbb{E}((X-c_{opt}) 1) = 0
$$

  where the $1$ represents the space of constants
we are projecting onto. By linearity of the expectation, gives

$$
c_{opt}=\mathbb{E}(X)
$$

 Using the projection approach, because $\mathbb{E}(X|Y)$ is
the projection operator, with $Y=\Omega$ (the entire underlying
probability space), we have, using the definition of conditional
expectation:

$$
\mathbb{E}(X|Y=\Omega) = \mathbb{E}(X)
$$

 This is because of the subtle fact that a random variable over the entire
$\Omega$ space can only be a constant.  Thus, we just worked the same problem
three ways (optimization, orthogonal inner products, projection).

**Example.** Let's consider the following example with probability density
$f_{X,Y}= x + y $ where $(x,y) \in [0,1]^2$ and compute the conditional
expectation straight from the definition:

$$
\mathbb{ E}(X|Y) = \int_0^1 x \frac{f_{X,Y}(x,y)}{f_Y(y)} dx=  \int_0^1 x \frac{x+y}{y+1/2} dx =\frac{3 y + 2}{6 y + 3}
$$

 That was pretty easy because the density function was so simple. Now,
let's do it the hard way by going directly for the MMSE solution $h(Y)$. Then,

$$
\mbox{ MSE } = \underset{h}\min \int_0^1\int_0^1 (x - h(y) )^2 f_{X,Y}(x,y)dx dy
$$

$$
\
             = \underset{h}\min \int_0^1 y h^2 {\left (y \right )} - y h{\left (y \right )} + \frac{1}{3} y + \frac{1}{2} h^{2}{\left (y \right )} - \frac{2}{3} h{\left (y \right )} + \frac{1}{4} dy
$$

 Now we have to find a function $h$ that is going to minimize this.
Solving for a function, as opposed to solving for a number, is generally very,
very hard, but because we are integrating over a finite interval, we can use
the Euler-Lagrange method from variational calculus to take the derivative of
the integrand with respect to the function $h(y)$ and set it to zero. Using
Euler-Lagrange methods, we obtain the following result,

$$
2 y h{\left (y \right )} - y + h{\left (y \right )} - \frac{2}{3} =0
$$

 Solving this gives

$$
h_{opt}(y)= \frac{3 y + 2}{6 y + 3}
$$

  which is what we obtained before. Finally, we can solve this
using our inner product in Equation ref{eq:ortho} as

$$
\mathbb{E}((X-h(Y)) Y)=0
$$

  Writing this out gives,

$$
\int_0^1\int_0^1 (x-h(y))y(x+y) dx dy = \int_0^1\frac{1}{6}y(-3(2 y+1) h(y)+3 y+2) dy=0
$$

 and the integrand must be zero,

$$
2 y + 3 y^2 - 3 y h(y) - 6 y^2 h(y)=0
$$

 and solving this for $h(y)$ gives the same solution:

$$
h_{opt}(y)= \frac{3 y + 2}{6 y + 3}
$$

 Thus, doing it by the brute force integration from the definition,
optimization, or inner product gives us the same answer; but, in general, no
method is necessarily easiest because they both involve potentially difficult
or impossible integration, optimization, or functional equation solving.  The
point is that now that we have a deep toolbox, we can pick and choose which
tools we want to apply for different problems.

Before we leave this example, let's use Sympy to verify the length of the error
function we found earlier for this example:

$$
\mathbb{E}(X-\mathbb{E}(X|Y))^2=\mathbb{E}(X)^2-\mathbb{E}(\mathbb{E}(X|Y))^2
$$

 that is based on the Pythagorean theorem. First, we
need to compute the marginal densities,

```python
from sympy.abc import y,x
from sympy import integrate, simplify
fxy = x + y                 # joint density
fy = integrate(fxy,(x,0,1)) # marginal density
fx = integrate(fxy,(y,0,1)) # marginal density
```

 Then, we need to write out the conditional expectation,

```python
EXY = (3*y+2)/(6*y+3) # conditional expectation
```

 Next, we can compute the left side, $\mathbb{E}(X-\mathbb{E}(X|Y))^2$,
as the following,

```python
# from the definition
LHS=integrate((x-EXY)**2*fxy,(x,0,1),(y,0,1))
LHS # left-hand-side
```

    -log(216)/144 + log(72)/144 + 1/12

 We can similarly compute the right side, $\mathbb{E}(X)^2-\mathbb{E}(\mathbb{E}(X|Y))^2$,
as the following,

```python
# using Pythagorean theorem
RHS=integrate((x)**2*fx,(x,0,1))-integrate((EXY)**2*fy,(y,0,1))
RHS # right-hand-side
```

    -log(216)/144 + log(72)/144 + 1/12

 Finally, we can verify that the left and right sides match,

```python
print simplify(LHS-RHS)==0
```

    True

In this section, we have pulled together all the projection and least-squares
optimization ideas from the previous sections to connect geometric notions of
projection from vectors in $\mathbb{R}^n$ to random variables. This resulted in
the remarkable realization that the conditional expectation is in fact a
projection operator for random variables.  Knowing this allows to approach
difficult problems in multiple ways, depending on which way is more intuitive
or tractable in a particular situation. Indeed, finding the right problem to
solve is the hardest part, so having many ways of looking at the same concepts
is crucial.

For much more detailed development, the book by Mikosch
[[mikosch1998elementary]](#mikosch1998elementary) has some excellent sections covering much of this
material with a similar geometric interpretation. Kobayashi
[[kobayashi2011probability]](#kobayashi2011probability) does too.  Nelson [[edward1987radically]](#edward1987radically) also
has a similar presentation based on hyper-real numbers.

## Appendix

We want to prove that we the conditional expectation is the
minimum mean squared error minimizer of the following:

$$
J= \min_h \int_{ \mathbb{R}^2 } \lvert X - h(Y) \rvert^2 f_{X,Y}(x,y) dx dy
$$

 We can expand this as follows,

$$
\begin{multline*}
J=\min_h \int_{ \mathbb{R}^2 } \lvert X \rvert^2 f_{X,Y}(x,y) dx dy + \int_{ \mathbb{R}^2 } \lvert h(Y) \rvert^2 f_{X,Y}(x,y) dx dy \\\
- \int_{ \mathbb{R}^2 } 2 X h(Y) f_{X,Y}(x,y) dx dy
\end{multline*}
$$

 To minimize this, we have to maximize the following:

$$
A=\max_h \int_{ \mathbb{R}^2 }  X h(Y) f_{X,Y}(x,y) dx dy
$$

 Breaking up the integral using the definition of conditional expectation

$$
\begin{equation}
A =\max_h \int_\mathbb{R} \left(\int_\mathbb{R} X  f_{X|Y}(x|y) dx \right)h(Y) f_Y(y) dy
\label{_auto1} \tag{3}
\end{equation}
$$

$$
\begin{equation} \
=\max_h \int_\mathbb{R} \mathbb{E}(X|Y) h(Y)f_Y(Y) dy
\label{_auto2} \tag{4}
\end{equation}
$$

 From properties of the Cauchy-Schwarz inequality, we know that the
maximum happens when $h_{opt}(Y) = \mathbb{E}(X|Y)$, so we have found the
optimal $h(Y)$ function as:

$$
h_{opt}(Y) = \mathbb{E}(X|Y)
$$

 which shows that the optimal function is the conditional expectation.

```python

```