# Inference Overview and Variable Elimination Algorithm

> So long, we have talked about probabilistic graphical model representation (PGM).
> - We've seen that there are two main types: Bayesian networks (directed) and Markov networks (undirected).
> - We've alse seen that they encode different **independence assumptions**.
>
> In this module, we will operationalize the PGMs and study how to use these representations (models) to answer actual queries or questions.

> **Objetives:**
> - To learn the different types of queries one can perform to a PGM.
> - To describe the Variable Elimination algorithm.
> - To analyze the computational complexity of the Variable Elimination algorithm.
> - To learn how to use the Variable Elimination algorithm to answer a query to an actual network.

> **References:**
> - Probabilistic Graphical Models: Principles and Techniques, By Daphne Koller and Nir Friedman. Ch. 9.
> - Mastering Probabilistic Graphical Models Using Python, By Ankur Ankan and Abinash Panda. Ch. 3.
> - Probabilistic Graphical Models Specialization, offered through Coursera. Prof. Daphne Koller.

 Imagen recuperada de: https://static.thenounproject.com/png/542457-200.png.

___

## 1. Inference - Overview

In the previous module, we looked at the different types of representations and how to use them to create models for some problems.

We also saw how the probabilities of variables change when we incorporate evidence, in an intuitive way.

In this module, we will describe several algorithms to compute these probabilities and how they may change. Similarly, we will see how to use inference algorithms to predict the values of variables based on our model.

### 1.1. Conditional probability queries

These are the most common queries. Given:

- A joint probability distribution

  $$P(X_1,\dots, X_n),$$

  (which is modeled through a BN or a MN);

- a set of query variables $\bar{Y}\subseteq \left\{X_1,\dots, X_n\right\}$;

- and a set of observed variables (evidence) $\bar{E} = \bar{e}$, con $\bar{E}\subseteq \left\{X_1,\dots, X_n\right\}$,

we define $\bar{W} = \left\{X_1,\dots, X_n\right\} \setminus \left\{\bar{Y} \cup \bar{E}\right\}$ as the rest of the variables.

The task is to compute the *conditional probability query* $P(\bar{Y} | \bar{E}=\bar{e})$.

**Applications:**

- In the credit-worthiness model of the exam, we saw that the bank was able to observe some variables. Then we would be interested in computing the probability of the unobserved variables given the evidence that the bank observes.

- In a medical diagnosis system, we observe certain symptoms and some test results and we are interested in computing the probability of different diseases.

By definition of conditional probability:

$$P(\bar{Y} | \bar{E}=\bar{e}) = \frac{P(\bar{Y}, \bar{e})}{P(\bar{e})}.$$

In this expression:

- $P(\bar{Y}, \bar{e}) = \sum_{\bar{W}} P(\bar{Y}, \bar{e}, \bar{W})$.

  Recall that $\left\{X_1,\dots, X_n\right\} = \bar{Y} \cup \bar{E} \cup \bar{W}$. Then, the terms in the right-side sumation are joint probabilities of all the variables.

- $P(\bar{e}) = \sum_{\bar{Y}} P(\bar{Y}, \bar{e})$.

  This is simply a normalizing constant for converting $P(\bar{Y}, \bar{e})$ into $P(\bar{Y} | \bar{e})$.

Hence, in principle, we could:

- Take a PGM;
- multiply all of its factors to obtain the joint distribution;
- sum out (marginalize) the unwanted variables in the joint distribution;
- and, that's it!

**Examples:**

1. Bayesian networks: restaurant example.

   Random variables:
   - Location $L$ (Bad: $l^0$, Good: $l^1$).
   - Quality $Q$ (Bad: $q^0$, Normal: $q^1$, Good: $q^2$).
   - Cost $C$ (Low: $c^0$, High: $c^1$).
   - Number of people $N$ (Low: $n^0$, High: $n^1$).

```python
from IPython.display import Image
```

```python
Image("figures/restaurant.png")
```

We will forget about if factors represent CPDs or general affinity functions.

In this case:

$$P(L,Q,C,N)= \phi_L(L)\phi_Q(Q)\phi_C(C,L,Q)\phi_N(N,L,C).$$

We'd like, for example:

$$P(N)=\sum_{L,Q,C}\phi_L(L)\phi_Q(Q)\phi_C(C,L,Q)\phi_N(N,L,C).$$

> Sum-product algorithm.

```python
# Import pgmpy.factors.discrete.DiscreteFactor
from pgmpy.factors.discrete import DiscreteFactor
```

```python
# Define factors
P_L = DiscreteFactor(variables=["L"],
                     cardinality=[2],
                     values=[0.4, 0.6])
P_Q = DiscreteFactor(variables=["Q"],
                     cardinality=[3],
                     values=[0.2, 0.5, 0.3])
P_C_given_LQ = DiscreteFactor(variables=["C", "L", "Q"],
                              cardinality=[2, 2, 3],
                              values=[0.95, 0.4, 0.4, 0.9, 0.4, 0.2,
                                      0.05, 0.6, 0.6, 0.1, 0.6, 0.8])
P_N_given_LC = DiscreteFactor(variables=["N", "L", "C"],
                              cardinality=[2, 2, 2],
                              values=[0.4, 0.9, 0.2, 0.4,
                                      0.6, 0.1, 0.8, 0.6])
```

```python
# Joint probability
joint = P_L * P_Q * P_C_given_LQ * P_N_given_LC
print(joint)
```

    +------+------+------+------+----------------+
    | L    | Q    | C    | N    |   phi(L,Q,C,N) |
    +======+======+======+======+================+
    | L(0) | Q(0) | C(0) | N(0) |         0.0304 |
    +------+------+------+------+----------------+
    | L(0) | Q(0) | C(0) | N(1) |         0.0456 |
    +------+------+------+------+----------------+
    | L(0) | Q(0) | C(1) | N(0) |         0.0036 |
    +------+------+------+------+----------------+
    | L(0) | Q(0) | C(1) | N(1) |         0.0004 |
    +------+------+------+------+----------------+
    | L(0) | Q(1) | C(0) | N(0) |         0.0320 |
    +------+------+------+------+----------------+
    | L(0) | Q(1) | C(0) | N(1) |         0.0480 |
    +------+------+------+------+----------------+
    | L(0) | Q(1) | C(1) | N(0) |         0.1080 |
    +------+------+------+------+----------------+
    | L(0) | Q(1) | C(1) | N(1) |         0.0120 |
    +------+------+------+------+----------------+
    | L(0) | Q(2) | C(0) | N(0) |         0.0192 |
    +------+------+------+------+----------------+
    | L(0) | Q(2) | C(0) | N(1) |         0.0288 |
    +------+------+------+------+----------------+
    | L(0) | Q(2) | C(1) | N(0) |         0.0648 |
    +------+------+------+------+----------------+
    | L(0) | Q(2) | C(1) | N(1) |         0.0072 |
    +------+------+------+------+----------------+
    | L(1) | Q(0) | C(0) | N(0) |         0.0216 |
    +------+------+------+------+----------------+
    | L(1) | Q(0) | C(0) | N(1) |         0.0864 |
    +------+------+------+------+----------------+
    | L(1) | Q(0) | C(1) | N(0) |         0.0048 |
    +------+------+------+------+----------------+
    | L(1) | Q(0) | C(1) | N(1) |         0.0072 |
    +------+------+------+------+----------------+
    | L(1) | Q(1) | C(0) | N(0) |         0.0240 |
    +------+------+------+------+----------------+
    | L(1) | Q(1) | C(0) | N(1) |         0.0960 |
    +------+------+------+------+----------------+
    | L(1) | Q(1) | C(1) | N(0) |         0.0720 |
    +------+------+------+------+----------------+
    | L(1) | Q(1) | C(1) | N(1) |         0.1080 |
    +------+------+------+------+----------------+
    | L(1) | Q(2) | C(0) | N(0) |         0.0072 |
    +------+------+------+------+----------------+
    | L(1) | Q(2) | C(0) | N(1) |         0.0288 |
    +------+------+------+------+----------------+
    | L(1) | Q(2) | C(1) | N(0) |         0.0576 |
    +------+------+------+------+----------------+
    | L(1) | Q(2) | C(1) | N(1) |         0.0864 |
    +------+------+------+------+----------------+

```python
# P(N)
P_N = joint.marginalize(variables=["L", "Q", "C"], inplace=False)
```

```python
# Show
print(P_N)
```

    +------+----------+
    | N    |   phi(N) |
    +======+==========+
    | N(0) |   0.4452 |
    +------+----------+
    | N(1) |   0.5548 |
    +------+----------+

2. Markov networks: pairwise.

```python
Image("figures/pairwiseMN.png")
```

$$
P(A, B, C, D) = \frac{1}{Z} \phi_1(A,B)\phi_2(B,C)\phi_3(C,D)\phi_4(A,D)
$$

For example

$$P(D) = \frac{1}{Z}\sum_{A,B,C}\phi_1(A,B)\phi_2(B,C)\phi_3(C,D)\phi_4(A,D).$$

> Sum-product algorithm.

**So, what is the problem?**

*There is an exponential blow-up of the joint distribution that the graphical model representation was precisely designed to avoid*.

In fact, the problem of inference in PGMs (just like most interesting problems) is $\mathcal{NP}-$hard. *What does this mean?*

- If a problem is $\mathcal{NP}-$hard it is very unlikely to come up with an efficient solution (in the general case).

- This means that **all the algorithms** that people have constructed to solve this problem require a time (or number of operations) which is at leat exponential in the size of the representation of the problem.

However, we will be seeing a variety of algorithms for both, exact and approximate inference that can do considerably better that this worst case result.

### 1.3. Maximum A-Posteriori (MAP) Inference

Given:

- A joint probability distribution

  $$P(X_1,\dots, X_n),$$

  (which is modeled through a BN or a MN);

- a set of query variables $\bar{Y}\subseteq \left\{X_1,\dots, X_n\right\}$;

- and a set of observed variables (evidence) $\bar{E} = \bar{e}$, con $\bar{E}\subseteq \left\{X_1,\dots, X_n\right\}$.

In this case, we assume for simplicity that $\bar{Y}\cup \bar{E} = \left\{X_1,\dots, X_n\right\}$.

The task is to compute

$$MAP(\bar{Y} | \bar{E}=\bar{e}) = \arg\max_{\bar{y}\in\mathrm{Val}(\bar{Y})} P(\bar{Y} | \bar{E}=\bar{e}).$$

> There may be multiple solutions!

**Application:**

- Classifier: Once we learn a model from data, we would like to compute the most likely assignment for a set of variables given some observed variables (evidence).

**MAP $\neq$ Max over marginals:**

```python
Image("figures/simple.png")
```

Again, this problem is $\mathcal{NP}-$hard.

___
## 2. Variable Elimination

This is the simplest and most fundamental algorithm.

### 2.1. Initial examples

We will introduce the **variable elimination (VE) algorithm** through a number of simple examples.

**Elimination in a chain**

Consider the following pairwise chain MN:

```python
Image("figures/MNchain.png")
```

We have that:

$$P(A,B,C,D,E) \propto \phi_1(A,B)\phi_2(B,C)\phi_3(C,D)\phi_4(D,E)$$

Our objective is to calculate $P(E)$ (in the whiteboard, then show):

$$\begin{align}P(E) \propto & \sum_{D,C,B,A} \phi_1(A,B)\phi_2(B,C)\phi_3(C,D)\phi_4(D,E) \\
                          = & \sum_{D}\sum_{C}\sum_{B}\sum_{A} \phi_1(A,B)\phi_2(B,C)\phi_3(C,D)\phi_4(D,E) \\
                          = & \sum_{D}\sum_{C}\sum_{B} \phi_2(B,C)\phi_3(C,D)\phi_4(D,E) \underbrace{\sum_{A} \phi_1(A,B)}_{\tau_1(B)} \\
                          = & \sum_{D}\sum_{C}\sum_{B} \phi_2(B,C)\phi_3(C,D)\phi_4(D,E) \tau_1(B) \\
                          = & \sum_{D}\sum_{C} \phi_3(C,D)\phi_4(D,E) \underbrace{\sum_{B} \phi_2(B,C)\tau_1(B)}_{\tau_2(C)} \\
                          = & \sum_{D}\phi_4(D,E) \underbrace{\sum_{C} \phi_3(C,D)\tau_2(C)}_{\tau_3(D)} \\
                          = & \sum_{D}\phi_4(D,E)\tau_3(D) = \tau_4(E)
\end{align}$$

**Elimination in a BN**

```python
Image("figures/restaurant.png")
```

The joint probability is:

$$P(L,Q,C,N)= \phi_L(L)\phi_Q(Q)\phi_C(C,L,Q)\phi_N(N,L,C).$$

If we want to calculate $P(N)$ (in the whiteboard, then show):

$$\begin{align} P(N) = & \sum_{L,C,Q}\phi_L(L)\phi_Q(Q)\phi_C(C,L,Q)\phi_N(N,L,C) \\
                    = & \sum_{L}\sum_{C}\sum_{Q} \phi_L(L)\phi_Q(Q)\phi_C(C,L,Q)\phi_N(N,L,C) \\
                    = & \sum_{L}\sum_{C} \phi_L(L)\phi_N(N,L,C) \underbrace{\sum_{Q} \phi_Q(Q) \phi_C(C,L,Q)}_{\tau_1(C,L)} \\
                    = & \sum_{L} \phi_L(L) \underbrace{\sum_{C}\phi_N(N,L,C)\tau_1(C,L)}_{\tau_2(N,L)} \\
                    = & \sum_{L} \phi_L(L)\tau_2(N,L) = \tau_3(N)
\end{align}$$

```python
# Import pgmpy.models.BayesianModel
from pgmpy.models import BayesianModel
```

```python
# Import pgmpy.factors.discrete.TabularCPD
from pgmpy.factors.discrete import TabularCPD
```

```python
# Define model skeleton
restaurant_model = BayesianModel([("L", "C"),
                                  ("Q", "C"),
                                  ("L", "N"),
                                  ("C", "N")])
```

```python
# Define CPDs
cpd_L = TabularCPD(variable="L",
                   variable_card=2,
                   values=[[0.4],
                           [0.6]])
cpd_Q = TabularCPD(variable="Q",
                   variable_card=3,
                   values=[[0.2],
                           [0.5],
                           [0.3]])
cpd_C = TabularCPD(variable="C",
                   variable_card=2,
                   evidence=["L", "Q"],
                   evidence_card=[2, 3],
                   values=[[0.95, 0.4, 0.4, 0.9, 0.4, 0.2],
                           [0.05, 0.6, 0.6, 0.1, 0.6, 0.8]])
cpd_N = TabularCPD(variable="N",
                   variable_card=2,
                   evidence=["L", "C"],
                   evidence_card=[2, 2],
                   values=[[0.4, 0.9, 0.2, 0.4],
                           [0.6, 0.1, 0.8, 0.6]])
```

```python
# Attach CPDs to the model
restaurant_model.add_cpds(cpd_L, cpd_Q, cpd_C, cpd_N)
```

```python
# Check if the model is correctly defined
restaurant_model.check_model()
```

    True

```python
# Import pgmpy.inference.VariableElimination
from pgmpy.inference import VariableElimination
```

```python
# Create inference object
inference = VariableElimination(restaurant_model)
```

```python
# Perform query
P_N = inference.query(variables=["N"])
```

    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1532.07it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 233.88it/s]

```python
# Show
print(P_N)
```

    +------+----------+
    | N    |   phi(N) |
    +======+==========+
    | N(0) |   0.4452 |
    +------+----------+
    | N(1) |   0.5548 |
    +------+----------+

**What happens if we have evidence?**

If we now want to calculate $P(N, L=l^1)$:

- First, we should reduce our factors according to the given evidence: $\phi_L(l^1)=\phi_L'()$, $\phi_C(C,l^1,Q)=\phi_C'(C,Q)$, $\phi_N(N,l^1,C)=\phi_N'(N,C)$.
- Then, we run the VE algorithm as usual:

$$\begin{align} P(N, l^1) = & \sum_{C,Q}\phi_L'()\phi_Q(Q)\phi_C'(C,Q)\phi_N'(N,C) \\
                         = & \sum_{C}\sum_{Q} \phi_L'()\phi_Q(Q)\phi_C'(C,Q)\phi_N'(N,C) \\
                         = & \sum_{C} \phi_L'()\phi_N'(N,C) \underbrace{\sum_{Q} \phi_Q(Q) \phi_C'(C,Q)}_{\tau_1(C)} \\
                         = & \sum_{C} \phi_L'()\phi_N'(N,C) \tau_1(C)
\end{align}$$

**And, if we want the conditional probability?**

You only have to take the above and renormalize:

$$P(N|l^1)=\frac{P(N,l^1)}{P(l^1)}.$$

```python
# Perform query
P_N_given_l1 = inference.query(variables=["N"],
                               evidence={"L": 1})
```

    Finding Elimination Order: : 100%|██████████| 2/2 [00:00<00:00, 698.47it/s]
    Eliminating: C: 100%|██████████| 2/2 [00:00<00:00, 364.45it/s]

```python
# Show
print(P_N_given_l1)
```

    +------+----------+
    | N    |   phi(N) |
    +======+==========+
    | N(0) |   0.3120 |
    +------+----------+
    | N(1) |   0.6880 |
    +------+----------+

### 2.2. Algorithm description and computational complexity

#### Algorithm description

For the algorithm description, we have a set of factors $\bar{\Phi}$.

In the examples above:

1. MN: $\bar{\Phi}=\{\phi_1(A,B), \phi_2(B,C), \phi_3(C,D), \phi_4(D,E)\}$.
2. BN: $\bar{\Phi}=\{\phi_L(L), \phi_Q(Q), \phi_C(C,L,Q), \phi_N(N,L,C)\}$.

We will describe the steps for eliminating some variable $Z$ from $\bar{\Phi}$: *Eliminate Var-$Z$ from $\bar{\Phi}$*.

1. Determine the set of factors that involve $Z$:

   $$\Phi' = \left\{\phi_i \in \Phi : Z  \in \mathrm{scope}[\phi_i]\right\}$$

2. Compute:

   $$\psi = \prod_{\phi_i \in \Phi'} \phi_i$$

3. Compute:

   $$\tau = \sum_Z \psi$$

4. Overwrite:

   $$\bar{\Phi} := \left(\bar{\Phi}\setminus \Phi'\right) \cup \{\tau\}$$

Thus, the complete algorithm can be described as:

1. The first step is to reduce all factors in $\bar{\Phi}$ acording to the given evidence, if any.

2. For each non-query variable $Z$:
   - Eliminate Var-$Z$ from $\bar{\Phi}$.

3. Multiply all the remaining factors.

#### Computational complexity

We analyze the computational complexity of Eliminate Var-$Z$ from $\bar{\Phi}$.

1. Determine the set of factors that involve $Z$:

   $$\Phi' = \left\{\phi_i \in \Phi : Z  \in \mathrm{scope}[\phi_i]\right\}$$

2. Compute:

   $$\psi = \prod_{\phi_i \in \Phi'} \phi_i$$

3. Compute:

   $$\tau = \sum_Z \psi$$

4. Overwrite:

   $$\Phi := \left(\Phi\setminus \Phi'\right) \cup \{\tau\}$$

**Complexity of the second step:** Assume that $|\Phi'| = m_k$ (the number of factors that involve $Z$ is $m_k$). Thus,

   $$\psi(\bar{X}_k) = \prod_{i=1}^{m_k} \phi_i.$$

   Now, assuming that $N_k = |\mathrm{Val}(\bar{X}_k)|$ (the number of rows in the factor resulting from the multiplication), the computational cost of this step is:

   $$N_k (m_k - 1)$$

   (each row in the resulting factor is produced by $m_k - 1$ multiplications). See this in the whiteboard

**Complexity of the third step:**

   $$\tau_k (\bar{X}_k \setminus \{Z\}) = \sum_Z \phi(\bar{X}_k)$$

   Assuming that $|\mathrm{Val}(Z)|=z$ (the number of possible values of the variable $Z$). The computational cost of this step is (Each row of the resulting factor involves $z-1$ sumations):

   $$N_k \frac{(z-1)}{z} \leq N_k.$$

   See this in the whiteboard

**The whole algorithm:**

1. At most, $n$ elimination steps ($n$ is the number of variables).

2. Start with $m = |\bar{\Phi}|$ factors.

3. At each elimination step, generate $1$ factor ($\psi$).

4. Total number of factors is $m^{\ast}\leq m+n$.

5. $N:=\max_k N_k$: size of the biggest factor.

6. Number of product operations (each factor is multiplied only once):

   $$\sum_k N_k(m_k-1)\leq N m^{\ast}.$$

7. Number of marginalization operation:

   $$\sum_k N_k\leq N n.$$

> Until here, the computational cost is linear in $N$ and in $m^\ast$.

8. However, $N_k=|\mathrm{Val}(\bar{X}_k)|=\mathcal{O}(d^{r_k})$, where
  - $d:=\max |\mathrm{Val}(X_i)|$;
  - $r_k=|\bar{X}_k|$.

> Exponential blowup!

**In the BN example**

- Operation: $\tau_1(C,L)=\sum_{Q}\phi_Q(Q)\phi_C(C,L,Q)$ - $r_1=3$.

- Operation: $\tau_2(N,L)=\sum_{C}\phi_N(N,L,C)\tau_1(C,L)$ - $r_2=3$.

- Operation: $\tau_3(N)=\sum_{L}\phi_L(L)\tau_2(N,L)$ - $r_3=2$.

On the other hand $d=3$. Total computational complexity $\leq (n+m^\ast) d^{r_k}=(4+7) 3^3 = 297$.

```python
# Perform query with given elimination order
%timeit inference.query(variables=["N"], elimination_order=["Q", "C", "L"])
```

    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 332.62it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 410.52it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 309.25it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 353.56it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 237.58it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 503.30it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 381.69it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 475.63it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 542.86it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 487.67it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 416.39it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 457.49it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 464.95it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 315.77it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 394.50it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 362.82it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 339.92it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 606.55it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 464.49it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 318.18it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 258.05it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 226.30it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 276.04it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 333.99it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 280.82it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 280.69it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 212.75it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 329.02it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 173.93it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 227.32it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 288.00it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 222.83it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 379.70it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 286.05it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 238.97it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 462.32it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 559.86it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 446.30it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 428.00it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 432.92it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 436.30it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 447.20it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 526.59it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 243.21it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 389.61it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 302.47it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 588.87it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 492.96it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 321.52it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 260.89it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 468.53it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 453.96it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 380.40it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 286.03it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 241.71it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 360.71it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 284.35it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 336.79it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 294.13it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 275.50it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 356.00it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 298.18it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 258.16it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 381.77it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 490.33it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 263.28it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 219.28it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 349.26it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 326.08it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 284.62it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 326.49it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 218.11it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 399.08it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 445.05it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 203.28it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 280.71it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 276.85it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 441.72it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 233.96it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 366.25it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 254.27it/s]

    22.3 ms ± 1.13 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

**A different elimination order**

$$\begin{align} P(N) = & \sum_{L,C,Q}\phi_L(L)\phi_Q(Q)\phi_C(C,L,Q)\phi_N(N,L,C) \\
                     = & \sum_{L}\sum_{C}\sum_{Q} \phi_L(L)\phi_Q(Q)\phi_C(C,L,Q)\phi_N(N,L,C) \\
                     = & \sum_{Q}\sum_{C} \phi_Q(Q) \underbrace{\sum_{L} \phi_L(L) \phi_C(C,L,Q) \phi_N(N,L,C)}_{\tau_1(Q,C,N)} \\
                     = & \sum_{Q} \phi_Q(Q) \underbrace{\sum_{C} \tau_1(Q,C,N)}_{\tau_2(Q,N)} \\
                     = & \sum_{Q} \phi_Q(Q)\tau_2(Q,N)
\end{align}$$

- Operation: $\tau_1(Q,C,N) = \sum_{L} \phi_L(L) \phi_C(C,L,Q) \phi_N(N,L,C)$ - $r_1=4$

- Operation: $\tau_2(Q,N) = \sum_{C} \tau_1(Q,C,N)$ - $r_2=3$

- Operation: $\tau_3(Q) = \sum_{Q} \phi_Q(Q)\tau_2(Q,N)$ - $r_3=2$

On the other hand $d=3$. Total computational complexity $\leq (n+m^\ast) d^{r_k}=(4+7) 3^4 = 891$.

```python
# Perform query with given elimination order
%timeit inference.query(variables=["N"], elimination_order=["L", "Q", "C"])
```

    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 268.45it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 347.81it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 344.29it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 286.59it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 550.27it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 400.27it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 232.45it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 468.13it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 303.33it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 244.23it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 292.54it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 224.76it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 355.05it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 250.11it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 266.38it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 239.12it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 372.62it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 199.97it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 410.13it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 346.81it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 307.35it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 335.01it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 263.74it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 255.98it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 313.24it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 314.74it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 266.55it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 305.89it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 278.54it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 421.75it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 331.77it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 234.24it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 360.80it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 259.20it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 277.36it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 316.22it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 283.25it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 230.91it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 342.58it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 236.73it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 360.31it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 298.82it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 300.32it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 255.33it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 226.40it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 243.28it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 245.29it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 302.53it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 416.76it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 312.72it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 162.89it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 352.79it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 254.68it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 249.99it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 218.61it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 224.73it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 253.09it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 349.06it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 219.38it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 330.90it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 292.54it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 342.43it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 191.15it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 230.68it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 254.90it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 310.57it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 350.53it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 210.73it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 301.34it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 309.44it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 242.79it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 328.66it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 369.88it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 247.23it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 386.32it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 401.64it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 310.14it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 279.58it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 200.39it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 229.38it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 389.44it/s]

    24 ms ± 368 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)

```python
# Show
P_N = inference.query(variables=["N"], elimination_order=["L", "Q", "C"])
print(P_N)
```

    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 239.77it/s]

    +------+----------+
    | N    |   phi(N) |
    +======+==========+
    | N(0) |   0.4452 |
    +------+----------+
    | N(1) |   0.5548 |
    +------+----------+

The computational complexity strongly depends on the elimination order!

The result is the same for any elimination order though.

### 2.3. Finding elimination orderings

The VE algorithm works well no matter the selected ordering. However, we have shown that the ordering significantly affects the computation complexity of the algorithm.

**How do we find a good elimination ordering?**

- Good ideas can be obtained from the graphical representation. We won't cover that here, but you can take a look at Section 9.4, pages 305-315.

In practice, one often performs **greedy search using heuristic cost functions** (at each iteration calculate the variable to eliminate to obtain the smallest cost).

> These algorithms are not optimal, but perform sufficiently well.

Possible cost functions:

- min-weight: # of values of factor formed ($N_k$).
- min-neighbors: # of resulting neighbor nodes after variable elimination.
- min-fill: # of new fill edges.

```python
# min-weight
inference.query(variables=["N"], elimination_order="MinWeight")
```

    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1788.87it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 380.65it/s]

    <DiscreteFactor representing phi(N:2) at 0x7fb9ab3ea9d0>

```python
# min-neighbors
inference.query(variables=["N"], elimination_order="MinNeighbors")
```

    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 705.52it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 327.27it/s]

    <DiscreteFactor representing phi(N:2) at 0x7fb9ac499bd0>

```python
# min-fill (default)
inference.query(variables=["N"], elimination_order="MinFill")
```

    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1933.75it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 247.69it/s]

    <DiscreteFactor representing phi(N:2) at 0x7fb9ac56ebd0>

```python
# Compare execution times using timeit
%timeit inference.query(variables=["N"], elimination_order="MinWeight")
```

    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 755.78it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 357.72it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1848.53it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 395.90it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 796.13it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 497.96it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1140.07it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 297.62it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1156.09it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 213.17it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1191.56it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 359.18it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1073.35it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 432.46it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1297.07it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 254.24it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 644.62it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 451.71it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1269.97it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 286.88it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 2328.88it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 371.25it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1135.23it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 240.78it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 809.76it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 347.86it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1318.27it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 234.55it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1246.20it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 281.55it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 499.52it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 265.47it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1616.09it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 272.43it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1400.12it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 270.83it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1686.72it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 307.30it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1681.98it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 403.62it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 889.13it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 427.66it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1243.49it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 270.73it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1218.80it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 405.56it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 776.39it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 281.34it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 604.08it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 378.02it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1771.74it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 423.85it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1114.81it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 191.88it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 2034.42it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 362.82it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 578.37it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 320.97it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 991.72it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 312.01it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1260.18it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 331.03it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 539.09it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 271.18it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 473.38it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 265.60it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1448.81it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 349.61it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1027.09it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 293.25it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1185.05it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 318.35it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1051.20it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 326.38it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 997.38it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 450.55it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1140.07it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 261.08it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 773.81it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 331.57it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 640.65it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 217.50it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 828.10it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 360.71it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1446.48it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 279.58it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 2323.71it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 349.63it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 791.68it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 279.98it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 797.30it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 270.17it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 795.93it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 279.99it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 738.35it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 336.02it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 843.64it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 262.66it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1428.09it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 316.36it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 914.72it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 256.32it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1608.86it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 265.66it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1240.18it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 255.50it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 612.25it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 249.28it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 998.56it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 292.69it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 945.80it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 429.73it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1008.25it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 343.70it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 840.54it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 408.43it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1345.05it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 333.01it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1166.27it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 340.44it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1974.41it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 443.09it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1577.40it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 248.92it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1271.00it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 546.96it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1440.19it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 330.15it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1803.74it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 385.20it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 987.90it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 282.64it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1214.45it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 303.83it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 878.94it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 324.03it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1139.03it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 368.10it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 661.21it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 419.25it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 2087.76it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 297.14it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 779.66it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 310.38it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 870.55it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 317.73it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1482.09it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 278.00it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1932.86it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 256.93it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 770.68it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 326.46it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 987.67it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 174.48it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 605.33it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 436.12it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 902.58it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 290.01it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 656.21it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 339.95it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1626.54it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 322.57it/s]

    31.6 ms ± 1.75 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

```python
%timeit inference.query(variables=["N"], elimination_order="MinNeighbors")
```

    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1432.81it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 403.69it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1533.19it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 257.47it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1097.89it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 277.25it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 462.98it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 425.92it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1107.94it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 382.31it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 862.97it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 339.90it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 522.72it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 237.99it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 944.45it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 294.48it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1109.90it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 451.58it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 753.51it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 377.24it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 889.19it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 219.89it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1035.63it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 271.66it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 2085.34it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 272.66it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 747.16it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 257.16it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1053.05it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 205.82it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1739.89it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 278.99it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 843.70it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 276.29it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1635.42it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 301.95it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1442.00it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 259.75it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1003.50it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 411.14it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1367.11it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 165.28it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1344.04it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 365.37it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1134.21it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 195.61it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1520.41it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 366.69it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 566.03it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 241.75it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1902.18it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 321.54it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1077.40it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 263.53it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 924.40it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 381.45it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 659.31it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 335.37it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 565.52it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 188.28it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1037.51it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 266.68it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1035.03it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 349.11it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1260.69it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 117.76it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1139.55it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 481.79it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1596.21it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 313.96it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1497.43it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 353.01it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 478.84it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 320.08it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 812.12it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 310.10it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 413.60it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 459.88it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1338.18it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 322.68it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 849.16it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 244.04it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 734.81it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 281.77it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 885.56it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 299.49it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1014.01it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 299.99it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1240.18it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 286.62it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 932.21it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 456.46it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1644.18it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 496.92it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 945.44it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 208.82it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1413.02it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 381.99it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 446.68it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 293.27it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 429.79it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 264.71it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1305.55it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 202.91it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1158.33it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 198.90it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1478.43it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 475.56it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1800.13it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 280.80it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1213.51it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 282.64it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1317.44it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 346.97it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1452.99it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 339.27it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 2406.37it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 318.29it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 891.58it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 129.48it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 826.41it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 298.30it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 790.98it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 322.26it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1314.69it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 298.93it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 646.90it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 263.35it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1585.35it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 297.77it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1360.61it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 421.65it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1742.79it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 339.89it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 881.09it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 330.74it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 906.16it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 258.07it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1265.50it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 283.05it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 741.83it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 232.78it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1110.98it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 272.22it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1585.15it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 203.68it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 326.98it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 316.60it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1129.83it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 167.24it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1540.14it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 334.84it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 689.32it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 265.67it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1464.32it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 365.95it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 677.81it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 242.02it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 962.22it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 198.71it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1193.15it/s]
    Eliminating: C: 100%|██████████| 3/3 [00:00<00:00, 233.22it/s]

    33.7 ms ± 1.5 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

```python
%timeit inference.query(variables=["N"], elimination_order="MinFill")
```

    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1667.72it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 269.30it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 829.08it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 325.59it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1252.65it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 341.71it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1140.79it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 329.12it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1768.50it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 242.31it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1180.50it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 337.60it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1104.64it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 328.24it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1015.41it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 298.44it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1360.46it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 341.19it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 882.76it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 257.07it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 764.08it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 432.36it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 534.08it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 433.50it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1354.02it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 388.21it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 731.39it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 382.40it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1764.78it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 296.28it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 850.94it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 381.43it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1166.49it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 279.08it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 601.88it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 296.84it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1433.79it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 361.55it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 2926.26it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 321.18it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 766.41it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 394.15it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1008.25it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 372.39it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1236.53it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 377.66it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 991.09it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 373.91it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1603.74it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 343.46it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 982.58it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 227.10it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1688.98it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 234.53it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 923.72it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 285.55it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1246.82it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 347.05it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1661.11it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 214.93it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 771.96it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 350.86it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1183.49it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 217.98it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1004.06it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 267.22it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 2016.82it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 347.12it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1802.71it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 331.95it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1095.88it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 340.32it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1007.36it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 360.65it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1120.17it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 219.74it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1411.27it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 256.37it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1114.72it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 265.60it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1394.85it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 287.75it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 672.02it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 337.01it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1366.22it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 321.97it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 2611.10it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 277.43it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 429.14it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 239.03it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 962.59it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 331.17it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 938.60it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 249.78it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 899.94it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 310.70it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1554.79it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 177.15it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1456.02it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 370.31it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1487.69it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 331.11it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1171.48it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 270.25it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1391.15it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 283.74it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 657.55it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 302.06it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1286.99it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 267.53it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1281.09it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 248.93it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 2230.22it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 376.55it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1522.07it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 348.62it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1423.25it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 297.32it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 966.88it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 446.58it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1387.62it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 292.39it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 970.83it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 288.94it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 890.45it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 421.92it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1071.25it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 215.86it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 2556.46it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 359.60it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 563.67it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 310.99it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1244.35it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 398.09it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 2202.51it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 231.93it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 800.34it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 255.17it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 2100.30it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 439.53it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 641.04it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 214.06it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1665.51it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 270.52it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1439.36it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 308.45it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1752.98it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 343.90it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 617.38it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 338.65it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 477.24it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 263.16it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1905.06it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 321.09it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1538.25it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 358.27it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 397.16it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 240.35it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 2467.72it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 263.49it/s]
    Finding Elimination Order: : 100%|██████████| 3/3 [00:00<00:00, 1479.47it/s]
    Eliminating: L: 100%|██████████| 3/3 [00:00<00:00, 265.41it/s]

    30.3 ms ± 640 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)

# Announcements

## 1. Quiz.

Created with Jupyter by Esteban Jiménez Rodríguez.


## 2-4. 位相推定アルゴリズム（入門編）

この節では、量子コンピュータの将来の応用を考える上でも非常に大切な、ユニタリ行列の固有値を求める**位相推定アルゴリズム**という量子アルゴリズムについて学ぶ。位相推定アルゴリズムは、数多くの量子アルゴリズムの基礎として使われており、量子アルゴリズムの中でも最も重要なものの一つと言える。
(参照：Nielsen-Chuang の `5.2 Phase estimation`)

### はじめに：アダマールテストを改良する
[2-2節](2.2_Hadamard_test.ipynb)と同様に、ユニタリ演算$U$の固有値$e^{i\lambda}$を求める問題を考えよう。アダマールテストでは、固有値の位相$\lambda$はテストの測定結果の確率分布に反映され、測定結果をたくさんサンプルすることで$\lambda$を推定していた。これをもう少し工夫することで、測定結果から位相の情報をより直接的に取り出すことができることを見ていこう。

下準備として、$\lambda/2\pi$を2進展開しよう：

$$
\frac{\lambda}{2\pi} = \frac{j_1}{2^1} + \frac{j_2}{2^2} +\cdots+ \frac{j_k}{2^k} +\cdots
$$

$j_k$は0または1の値を取る（古典）ビットである。$\lambda$は$e^{i\lambda}$の形でのみ出てくるので、$0 \leq \lambda < 2\pi$として一般性を失わない。この2進展開を、通常の小数の表記にならって以下のように書く。

$$
\lambda = (2\pi) 0.j_1 j_2 \ldots j_k \ldots
$$

以下では簡単のため、$\lambda/2\pi$は小数点以下$n$桁で書けるものとする。

$$
\lambda = (2\pi) 0.j_1 j_2 \ldots j_n
$$

さて、アダマールテストでは制御ユニタリ演算として$\Lambda(U)$を用いたが、ここではそれを少し変えて$\Lambda(U^{2^k})$としてみよう。[2.2節](2.2_Hadamard_test.ipynb)の前半と同様、$|\psi\rangle$が$U$の固有状態であることを仮定する。制御ユニタリ演算を行なった後の状態は

$$
\frac{1}{\sqrt{2}}
\left(
 |0\rangle +
 e^{i2^k\lambda} |1\rangle
\right)
\otimes |\psi\rangle
$$

上記の2進展開を使うと、

$$
2^k \lambda =
2^k \cdot (2\pi)0.j_1 j_2 \ldots j_r =
(2\pi)j_1 j_2 \ldots j_k . j_{k+1} \ldots j_n
$$

$e^{i(2\pi)j_1 \ldots j_k} = 1$だから、結局、

$$
\frac{1}{\sqrt{2}}
\left(
 |0\rangle +
 e^{i(2\pi)0.j_{k+1} \ldots j_n} |1\rangle  \tag{1}
\right)
$$

となる。

まず、$k=n-1$のときを考えよう。このとき、

$$
\frac{1}{\sqrt{2}} \left( |0\rangle + e^{i(2\pi)0.j_n} |1\rangle \right)
$$

だから、アダマールゲートをかければ

$$
\frac{1}{\sqrt{2}} \left( |0\rangle + e^{i(2\pi)0.j_n} |1\rangle \right) \to |j_n\rangle
$$

と、$\lambda$の二進小数表示の$n$番目のビット$j_n=\pm 1$に対応した状態に変換できる。この状態を測定すれば、100%の確率で$j_n$が観測されるので、1回の測定で$\lambda$の$n$桁目を決定することができる。
次に、$k=n-1$の時を考えると、状態は

$$
\frac{1}{\sqrt{2}} \left( |0\rangle + e^{i(2\pi)0.j_{n-1}j_n} |1\rangle \right)
$$

である。$j_n$は先ほど調べてあるので、$j_n=0$の時は何もせず、$j_n=1$の時は一般位相ゲート

$$
R_{l}^\dagger = \begin{pmatrix} 1 & 0\\ 0 & e^{-i \frac{2\pi}{2^l} } \end{pmatrix}
$$

の$R_2$をかけると、

$$
\frac{1}{\sqrt{2}} \left( |0\rangle + e^{i(2\pi)0.j_{n-1}j_n} |1\rangle \right) \to \frac{1}{\sqrt{2}} \left( |0\rangle + e^{i(2\pi)0.j_{n-1}} |1\rangle \right)
$$

と変換できる。そしてアダマールゲートをかければ

$$
\frac{1}{\sqrt{2}} \left( |0\rangle + e^{i(2\pi)0.j_{n-1}} |1\rangle \right) \to |j_{n-1}\rangle
$$

となるから、$j_{n-1}$もこの状態を1回測定するだけで決定できる。
以下、同様に$k=n-2, n-3, \cdots, 1$と下の方の桁から$j_k$を決定していくことができる。

このように、アダマールテストを少し変形することで、固有値の位相を1桁ずつ(確定した)量子ビットの状態として取り出すことができる。この手続きを量子回路でいっぺんに行うのが、以下で説明する位相推定アルゴリズムである。

### 位相推定アルゴリズム：概要
上記のアダマールテスト改良版で出てきた式(1)は、どこかで見たことがある形ではないだろうか。そう、[2-3節](2.3_quantum_Fourier_transform.ipynb)で学んだ量子フーリエ変換の途中式(`回路の構成`セクション)と同じである。実は、上記のアルゴリズムの測定側の量子ビットを拡張し、量子フーリエ変換を組み合わせたのが、Kitaevによって提案された位相推定アルゴリズム[1]である。詳細はともかく、どのような操作ができるアルゴリズムなのかを、まず紹介しよう。

$U$を量子回路として構成できる一般的なユニタリ行列とする。$U$の固有ベクトルを$|{\rm eigen}_l \rangle$とし、対応する固有値を$e^{i\lambda_l}$とする。ある一般的な量子状態$|\psi\rangle$が与えられたとする。これは必ず固有ベクトルで展開できる：

$$
|\psi \rangle = \sum _l c_l |{\rm eigen}_l \rangle
$$

もちろん具体的に係数$c_l$がどのような値になるかはわからなくてよい。このとき**位相推定アルゴリズムは、**$n$**個の補助量子ビットを用いて、入力状態**

$$
|00...0\rangle |\psi\rangle
$$

**を、**

$$
\sum _l c_l |\lambda_l \rangle | {\rm eigen}_l \rangle
$$

**へと変換するアルゴリズム**のことである。ここで、$|\lambda_l\rangle$ は固有値の位相$\lambda_l$の二進小数表示$\lambda_l=(2\pi)0.j_1^{(l)}\cdots j_n^{(l)}$に対応する量子状態$|j_1^{(l)}\cdots j_n^{(l)}\rangle$である
（ここでは、$\lambda_l$の二進小数表示が$n$桁で終わるとすると仮定した；次節の記述も参照）。
つまり位相推定アルゴリズムは、$|\psi\rangle$の重ね合わせの中にある**それぞれの固有ベクトルに対応した固有値を**$n$**個の補助量子ビットへと取り出すアルゴリズム**になっている。この状態に対して補助量子ビットの測定をすると、確率

$$
p_l = |c_l |^2
$$

で、どれか一つの固有ベクトル$|{\rm eigen}_l\rangle$とその固有値$\lambda_l$が乱択される。このアルゴリズムは、素因数分解や量子化学アルゴリズム(分子などのエネルギー計算)、そしてその他多くのアルゴリズムのサブルーチンとして利用されていおり、**量子コンピュータが従来コンピュータよりも指数的に高速に特定の問題において解を得られる（と期待されている）最も重要な例**である。

### 位相推定アルゴリズム：構成
以下では、入力状態$|\psi\rangle$を固有状態$|{\rm eigen}\rangle$とその固有値$\lambda$に限定して、位相推定アルゴリズムを説明していくことにする（入力状態が固有状態の重ね合わせの場合でも全く同じ議論が使えるので、一般性は失われていない）。 アダマールテストでは1つしか測定用の量子ビットを使わなかったが、位相推定では、測定用の補助量子ビットとして$n$個の量子ビットを確保する。
位相推定を行う回路は以下の図のようである。

再び、ユニタリー演算$U$の固有値$e^{i\lambda}$の位相$\lambda$を$n$ビットの二進小数を用いて

$$
\lambda = (2\pi) 0.j_1 j_2...j_n
$$

と書いておく。（$\lambda$の二進小数表示が$n$桁で終わると仮定する；終わらない場合には、最後の測定の際に若干のエラーが生じるが、測定を繰り返せばこのエラーは克服できる。詳細は冒頭のNielsen-Chuangの参照箇所にあたられたい）

1. まず、$|0\rangle$に初期化された$n$個の量子ビットのそれぞれに、アダマールテストと同様にアダマールゲートと制御ユニタリー演算を作用させる。ただし$k$番目（$k=1,...,n$）の補助量子ビットには制御$U^{2^{k-1}}$演算をすることにする。
$U|{\rm eigen}\rangle = e^{i2\pi\lambda}|{\rm eigen}\rangle$だから、$k$番目の補助量子ビットには$e^{i \lambda 2^k}$の位相が獲得される（これを位相キックバックと呼ぶ）ことになり、

    $$
\left( \frac{|0\rangle + e^{i (2\pi)0.j_1\cdots j_n} |1\rangle }{\sqrt{2}} \right) \otimes
\left( \frac{|0\rangle + e^{i (2\pi)0.j_2\cdots j_n} |1\rangle }{\sqrt{2}} \right) \otimes \cdots \otimes
\left( \frac{|0\rangle + e^{i (2\pi)0.j_n} |1\rangle }{\sqrt{2}} \right) \otimes
|{\rm eigen} \rangle
$$

    という状態が得られる。
つまり、固有値の位相を二進小数表示で1ビットずつシフトしたものが各補助量子ビットの位相に格納されることになる。

2. $n$個の補助量子ビットの状態は、[2-3節](2.3_quantum_Fourier_transform.ipynb)で学んだ量子フーリエ変換の結果の式と全く同じ形をしている。よって、逆量子フーリエ変換(図の$QFT^\dagger$)をこれらの補助量子ビットに作用させると、

    $$
\left( \frac{|0\rangle + e^{i (2\pi)0.j_1\cdots j_n} |1\rangle }{\sqrt{2}} \right) \otimes
\left( \frac{|0\rangle + e^{i (2\pi)0.j_2\cdots j_n} |1\rangle }{\sqrt{2}} \right) \otimes \cdots \otimes
\left( \frac{|0\rangle + e^{i (2\pi)0.j_n} |1\rangle }{\sqrt{2}} \right) \otimes
\rangle
\rightarrow |j_1...j_n\rangle
$$

    となる。よって、この段階で補助量子ビットの測定を行えば、100%の確率で $j_1, j_2,\cdots,j_n$ が得られ、$U$の固有値の位相$\lambda$が求められる。

まとめると、測定用の各量子ビットを制御ビットとする制御$U^{2^k}$演算を行なって**固有値の位相の情報を補助量子ビットに移した後、逆量子フーリエ変換で位相の値を取り出す**のが、位相推定アルゴリズムである。

### SymPyによる具体例
SymPyで具体例を見ていこう。T演算とS演算を用いて以下のような4×4行列$U$をつくり、この行列の固有値を求めてみる。

```python
from sympy import *
from sympy.physics.quantum import *
from sympy.physics.quantum.qubit import Qubit,QubitBra
init_printing() # ベクトルや行列を綺麗に表示するため
from sympy.physics.quantum.gate import X,Y,Z,H,S,T,CNOT,SWAP,CPHASE,CGateS
```

```python
# Google Colaboratory上でのみ実行してください
from IPython.display import HTML
def setup_mathjax():
    display(HTML('''

    '''))
get_ipython().events.register('pre_run_cell', setup_mathjax)
```

```python
represent(T(0)*S(1),nqubits = 2)
```

この行列はすでに対角化されており、固有値の位相$\lambda$は$0, \pi/4, \pi/2, 3\pi/4$で、その二進小数表示は$(2\pi)0.0, (2\pi)0.001, (2\pi)0.01, (2\pi)0.011$である。よってこれらを誤差なく測定するためには３つの補助量子ビットが必要である。
` CGateS ` 関数を用いて、$U$を制御化した `cP_2,3,4` を定義しよう。（2,3,4が３つの補助量子ビットに対応している。0,1は$U$の作用する空間である。先ほどの位相推定の回路図でいうと、下から順番に0,1,2,...とビットを名付けている。）

```python
cP_2 = CGateS(2,T(0))*CGateS(2,S(1))
cP_3 = CGateS(3,T(0))*CGateS(3,S(1))
cP_4 = CGateS(4,T(0))*CGateS(4,S(1))
```

３つの補助量子ビットを用いて、位相推定アルゴリズムを構成していこう。
まずはアダマールゲートを全ての補助量子ビットにかける。

```python
PhaEst = H(4)*H(3)*H(2)
PhaEst
```

次に、制御ユニタリーをそれぞれ、1回、２回、４回と作用させていく。

```python
PhaEst = cP_2*cP_3*cP_3*cP_4*cP_4*cP_4*cP_4*PhaEst
```

これで位相推定アルゴリズムの前半部分は終了である。ここから逆量子フーリエ変換を構築する。
まず小数第3位については、補助量子ビット4に$H$演算を作用させればよい。

```python
PhaEst = H(4)*PhaEst
```

補助量子ビット3には、まず補助量子ビット4を制御とする制御$R_2^\dagger=S^{-1}=SZ$演算を作用させる。

```python
PhaEst = CGateS(4,S(3))*PhaEst
PhaEst = CGateS(4,Z(3))*PhaEst
```

その後に$H$演算を作用させる。

```python
PhaEst = H(3)*PhaEst
```

補助量子ビット2には、

- 補助量子ビット3を制御とする制御$R_2^\dagger = S^{-1} = SZ$演算
- 補助量子ビット4を制御とする制御$R_3^\dagger = TS^{-1} = TSZ$演算
- $H$演算

の3つを作用させる。

```python
PhaEst = CGateS(3,S(2))*PhaEst
PhaEst = CGateS(3,Z(2))*PhaEst
```

```python
PhaEst = CGateS(4,T(2))*PhaEst
PhaEst = CGateS(4,S(2))*PhaEst
PhaEst = CGateS(4,Z(2))*PhaEst
PhaEst = H(2)*PhaEst
```

このように構成した位相推定アルゴリズムを固有ベクトルに作用させてみよう。アルゴリズム自体は非常に複雑だ。

```python
PhaEst
```

しかし実際、入力に作用させると、

```python
simplify(qapply(PhaEst*Qubit("00011")))
```

のように設計通り単純な解が得られることが確認できる。
実際、入力$|\psi\rangle$は$|11\rangle$なので対応する固有値は$e^{i5\pi/4}$だが、補助量子ビット2,3,4は011となっており、固有値の位相$\lambda=5\pi/4$の二進小数0.011が得られている！他の入力に対しても、

```python
simplify(qapply(PhaEst*Qubit("00000")))
```

```python
simplify(qapply(PhaEst*Qubit("00010")))
```

```python
simplify(qapply(PhaEst*Qubit("00001")))
```

のように、４種類の固有値の位相情報が３つの補助量子ビットに得られていることが確かめられる。
また、入力状態が重ね合わせ状態の場合は、

```python
simplify(qapply(PhaEst*H(0)*H(1)*Qubit("00000")))
```

のように、それぞれの固有ベクトルに対して、固有値の位相が３つの補助量子ビットへと重ね合わせのまま取り出されている。この状態の補助量子ビットを測定すると、確率的にどれか一つの固有ベクトルと固有値が得られる。

---
### コラム：素因数分解と位相推定（やや難）

位相推定の重要な応用例として、素因数分解アルゴリズムを紹介する。
素因数分解問題は$n$桁の整数$N$が与えられた時に$N$の1ではない約数を
見つける問題であり、従来のコンピュータでは多項式時間で解けるアルゴリズムは
見つかっていない。現在のベストアルゴリズムの計算コストは、
\begin{eqnarray}
O\left(\exp \left[\frac{64}{9} n (\log n )^2 \right]^{1/3}\right)
\end{eqnarray}
であり、準指数的な計算時間がかかる。
このような素因数分解問題の難しさを利用したRSA暗号などが日常的にも使われている。

P.\ Shorは1994年に、
量子コンピュータを用いることによって素因数分解問題が桁数$n$に対して多項式時間で解くことができる
ことを示した。これがいわゆるShorの素因数分解アルゴリズムである。

$N$を素因数分解したい整数だとしよう。まず、$N$と互いに素な整数$x$を見つけてくる。
（$x$はユークリッドの互除法で簡単に見つけられる。
適当に$x$を取ってきて、ユークリッドの互除法で$N$と$x$の最大公約数を計算し、
それが1以外の整数であれば$N$の非自明な約数がみつかったことになり(素)因数分解が完了するし、
1しかなければ$x$は$N$と互いに素な整数であることになる。）
このとき、$x$の$N$に関する位数$r$を考えてみる。
位数$r$とは、
\begin{eqnarray}
x^r \equiv 1 \textrm{ (mod $N$)}
\end{eqnarray}
を満たす最小の整数であり、ランダムに$x$を選ぶと高い確率で$r$は偶数になることが知られている。
$r$が偶数であると、上式は
\begin{eqnarray}
(x^{r/2} + 1) (x^{r/2} - 1)  \equiv 0  \textrm{ (mod $N$)}
\end{eqnarray}
のように変形することができる。
これはつまり、$x^{r/2} \pm 1 \equiv 0 \: (\textrm{mod} \: N)$であるか、
$x^{r/2} + 1$と$x^{r/2} - 1$が$N$と非自明な公約数($N$**の因数**)をもつかのどちらかであることを意味する。
実は、$x$がランダムに選ばれているとき、後者の確率が十分高いことも示すことができる。よって、$x^{r/2} + 1$か$x^{r/2} - 1$と$N$との公約数をユークリッドの互除法で用いることにより、最終的に非自明な$N$の因数が見つかることになる。
これを繰り返していくことによって、$N$をどんどん小さな因数へと分解していくことができ、最終的に素因数分解が達成できる。

そして、素因数分解の鍵となる位数$r$は、実は入力$y$をmod Nのもとでx倍するという古典計算に対応したユニタリ行列

$$
U_x = \sum _y |yx \: {\rm (mod N)}\rangle \langle y|
$$

の固有値を求めることによって決定することができる。実際、固有状態のラベル$0 \leq s \leq r-1$を用いて，固有ベクトルは

$$
|u_s\rangle = \frac{1}{\sqrt{r}}
\sum _{k=0}^{r-1} e^ {-2 \pi i (s/r) k }|x^k (\textrm{mod} cu)\rangle.
$$

と書き下すことができ、

$$
U_x |u_s \rangle =  e^{2 \pi i (s/r)} |u_s \rangle
$$

を満たす。つまり固有値の位相推定から$s/r$を求めその分母として$r$を得ることができる（そして、その位数$r$から上記の手順で$N$の素因数分解ができる）。
これがいわゆる、ショアによる素因数分解アルゴリズム（キタエフの位相推定を利用したバージョン）である。

(詳細は Nielsen-Chuang の `5.3 Applicaitons: order-finding and factoring` を参照)

---
### 第２章のまとめ
第2章では、以下のことを学んだ。

- 量子アルゴリズムとは何か
- NISQアルゴリズム
- アダマールテスト
- 量子フーリエ変換
- 位相推定アルゴリズム

この章で学んだアルゴリズム、特に位相推定アルゴリズムは量子コンピュータを使いこなす上で非常に大切なもので、様々な応用がある。

1. 素因数分解問題：上記コラムで見たように、位相推定アルゴリズムを用いて位数という数を推定することによって素因数分解を行うことができる。
2. 量子系のエネルギー計算：量子力学において、エネルギーはハミルトニアンと呼ばれる行列の固有値によって与えられる。もっとも安定的な状態のエネルギーはもっとも小さい値の固有値、安定な状態は対応する固有ベクトルで与えられる。ハミルトニアン自体はユニタリー行列ではないものの、行列の指数関数で定義される時間発展に対応するユニタリ行列$e^{-i H t}$に対して位相推定を行うことによって、エネルギーを計算することができる。
3. 連立一次方程式$Ax=b$の求解：位相推定を使って求めた固有値・固有ベクトルを用いることにより、連立一次方程式の解([7-3節])を求めることができる。また、その機械学習への応用も研究されている。


# Detectors Comparison

O Detectors Comparison é um software coletor de estatísticas de algoritmos de detecção e extração de keypoints em fotografias turisticas.
Os dados estatísticos coletados são utilizados para avaliar o desempenho e precisão dos algoritmos: [ORB](), [BRISK](), [AKAZE](), [SIFT]() e [SURF]() em relação ao tempo de execução, quantidade de keypoints, quantidade de matches e porcentagem de acerto. A comparação é dividida em quatro categorias/casos/situações de pares de fotos: retratando o mesmo objeto na mesma escala, o mesmo objeto em escalas diferentes, objetos diferentes na mesma escala e objetos diferentes em escalas diferentes. Todos os pares de imagens se encontram relativamente no mesmo ângulo de visão.

Para realizarmos a tarefa proposta utilizamos como biblioteca principal a [OpenCV](), a qual fornece os algoritmos comparados. A demais bibliotecas ([NumPy](), [SciPy](), [SymPy](), [Time]() e [Matplotlib]()) funcionam como auxiliáres para a coleta de dados. Os dados, por sua vez, são salvos pelo [SQLite 3]().

```python
import cv2
import sqlite3
import numpy as np
from scipy import stats
from sympy import Point, Line
from time import time, strftime
from matplotlib import pyplot as plt
```

## Processo de coleta de dados

A etapa principal do processo de coleta de cados consiste em:

+ Encontrar os Keypoints;
+ Encontrar os Matches através de uma busca completa;
+ Avaliar a taxa de acerto;
+ Calcular os ângulos entre as retas que passam pelo centro da imagem e cada keypoint com a reta horizontal (que passa pelo centro da imagem e o pixel mais à direita de mesma altura/coordenada y);
+ Calcular as diferenças entre os ângulos dos keypoints;
+ Calcular as razões entre as distâncias dos centros das imagens e seus keypoints, que chamamos de escala entre as imagens;
+ Calcular as médias e desvios padrão dos ângulos dos keypoints e das escalas;
+ Rotacionar a imagem da esquerda com a média dos ângulos dos keypoints;
+ Ampliar a imagem da esquerda com a escala;
+ Rencontrar os novos keypoints e matches;
+ Reavaliar a taxa de acerto;
+ Remover os falsos Matches:
    + Filtrar os ângulos e escalas menores que a média menos um desvio padrão ou maiores que a média mais um desvio padrão.
+ Recalcular as médias e os desvios padrão dos ângulos dos keypoints e das escalas;
+ Reaplicar a rotação e a ampliação com os novos valores de média;
+ Rencontrar os novos keypoints e matches;
+ Reavaliar a taxa de acerto;

```python
NUM_OF_PAIRS = 1
TABLE_NAME = 'datas_{}'.format(strftime('%y%m%d_%H%M%S'))
```

```python
# Finds the image's center
def image_center(image):
    return Point(image.shape[1] / 2, image.shape[0] / 2)
```

```python
# Finds the angles between the horizontal axis
# and the lines passing through the image center
# and each keypoint
def g_find_kp_angles(image, kps):
    angles = []
    center = image_center(image)
    h_axis = Line(center, center.translate(center.x))
    for kp in kps:
        p = Point(kp.pt[0], kp.pt[1])
        kp_line = Line(center, p)
        angles.append(float(h_axis.angle_between(kp_line)))
    return angles
```

```python
def angles_dif(angles_img1, angles_img2, matches):
    dif = []
    for match in matches :
        dif.append(angles_img1[match.queryIdx] - angles_img2[match.trainIdx])

    return dif
```

```python
def remove_fake_matches(matches,dif_angles,angles_mean,angles_std,scales,scale_mean,scale_std):
    new_scales,new_dif_angles = [],[]
    for i in range(len(matches)):
        if dif_angles[i] < angles_mean + angles_std and dif_angles[i] > angles_mean - angles_std and scales[i] < scale_mean + scale_std and scales[i] > angles_mean - scale_std:
            new_scales.append(scales[i])
            new_dif_angles.append(dif_angles[i])
    return new_dif_angles,new_scales
```

```python
# Finds the Key's points Angles
def find_kp_angles(kp1, kp2, matches, center1, center2):
    central_line = Line(center1, center2.translate(2 * center2.x))
    angles = []
    for match in matches:
        p1 = Point(kp1[match.queryIdx].pt[0], kp1[match.queryIdx].pt[1])
        p2 = Point(kp2[match.trainIdx].pt[0], kp2[match.trainIdx].pt[1])
        match_line = Line(p1, p2.translate(2 * center2.x))
        angles.append(float(central_line.angle_between(match_line)))
    return angles
```

```python
def g_find_scale(image, kps):
    scale = []
    center = image_center(image)
    for kp in kps:
        p = Point(kp.pt[0], kp.pt[1])
        d = center.distance(p)
        scale.append(d)
    return scale

```

```python
# Finds the ratio of the keypoints scale between images
def find_scale_ratios(img1, kp1, img2, kp2, matches):
    ratios = []
    scale1 = g_find_scale(img1, kp1)
    scale2 = g_find_scale(img2, kp2)
    for match in matches:
        # scale list preserves the ordering from keypoints list
        d1 = scale1[match.queryIdx]
        d2 = scale2[match.trainIdx]
        ratios.append(float(d1 / d2))
    return ratios
```

```python
# Finds the Scale between images
def find_scale(kp1, kp2, matches, center1, center2):
    scale = []
    for match in matches:
        p1 = Point(kp1[match.queryIdx].pt[0], kp1[match.queryIdx].pt[1])
        p2 = Point(kp2[match.trainIdx].pt[0], kp2[match.trainIdx].pt[1])
        d1 = center1.distance(p1)
        d2 = center2.distance(p2)
        scale.append(float(d1 / d2))
    return scale
```

```python
def affine_trans(img,angles,scale):
    center = image_center(img)
    m = cv2.getRotationMatrix2D((center.y, center.x), angles, scale)
    return cv2.warpAffine(img, m, (img.shape[1],img.shape[0]))
```

```python
def save(conn,cursor,values):
    cursor.execute("""
      INSERT INTO {} (kp1,kp2,matches,time,anglesMean,anglesSD,scaleMean,scaleSD,technique,situation,pathImg1,pathImg2,phase)
      VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?)
      """.format(TABLE_NAME), values)
    conn.commit()
```

```python
def ploting_image_pair(left,right):
    fig = plt.figure()
    fig.add_subplot(1,2,1)
    plt.imshow(left)
    fig.add_subplot(1,2,2)
    plt.imshow(right)
    plt.show()
```

```python
def getStats(method,img1, img2):
    timeI = time()
    # find the keypoints and descriptors with ORB
    kp1, des1 = method.detectAndCompute(img1, None)
    kp2, des2 = method.detectAndCompute(img2, None)
    timeF = time()

    # create BFMatcher object
    bf = cv2.BFMatcher(cv2.NORM_L2, crossCheck=True)

    # Match descriptors. (query,train)
    matches = bf.match(des1, des2)

    # Sort them in the order of their distance.
    matches = sorted(matches, key=lambda x: x.distance)

    return [kp1,kp2, matches, timeF - timeI]

```

```python
def prep_values(img1,img2,method,name,case,pair):
    values = getStats(method,img1,img2)
    kp1,kp2,matches = values[0],values[1],values[2]
    values[0],values[1],values[2] = len(kp1),len(kp2),len(matches)

    angles_img1 = g_find_kp_angles(img1,kp1)
    angles_img2 = g_find_kp_angles(img2,kp2)
    angles_dif = angles_dif(angles_img1,angles_img2,matches)
    scales =  find_scale_ratios(img1, kp1, img2, kp2, matches)

    angles_mean = stats.tstd(angles_dif)
    angles_std = stats.tstd(angles_dif)

    scale_mean = stats.tmean(scales)
    scale_std = stats.tstd(scales)

    values.append(angles_mean)
    values.append(angles_std)
    values.append(scale_mean)
    values.append(scale_std)
    values.append(name)
    values.append(case)
    values.append('{}a.jpg'.format(pair))
    values.append('{}b.jpg'.format(pair))

    return angles_dif,scales,matches, values
```

```python
def main():
    executeTimeI = time()
    conn = sqlite3.connect('banco.db')
    cursor = conn.cursor()
    cursor.execute(
      """CREATE TABLE {} (
            technique TEXT,
            situation TEXT,
            kp1 INTEGER,
            kp2 INTEGER,
            matches INTEGER,
            time FLOAT,
            anglesMean FLOAT,
            anglesSD FLOAT,
            scaleMean FLOAT,
            scaleSD FLOAT,
            pathImg1 TEXT,
            pathImg2 TEXT,
            phase INTEGER
      );""".format(TABLE_NAME)
    )

    # Initiate detectors
    # SIFT = cv2.xfeatures2d.SIFT_create()
    # SURF = cv2.xfeatures2d.SURF_create()
    ORB = cv2.ORB.create()
    # # KAZE = cv2.KAZE.create()
    # AKAZE = cv2.AKAZE.create()
    # BRISK = cv2.BRISK.create()

    methods = {
        # 'SIFT': SIFT,
        # 'SURF': SURF,
        'ORB': ORB,
        # 'KAZE': KAZE,
        # 'AKAZE': AKAZE,
        # 'BRISK': BRISK
    }

    cases = [
        'Same Object, Same Scale',
        # 'Same Object, Different Scale',
        # 'Different Object, Same Scale',
        # 'Different Object, Different Scale',
    ]

    for case in cases:
      print(case)
      for pair in range(NUM_OF_PAIRS):
        print('Pair {}/{}'.format(pair + 1, NUM_OF_PAIRS))
        img1 = cv2.imread('photos/{}/{}a.jpg'.format(case,pair),0)
        img2 = cv2.imread('photos/{}/{}b.jpg'.format(case,pair),0)
        for name, method in methods.items():
          print(name)
          print("Phase One: Compares unaltered images")
          angles_dif,scales,matches,original_values = prep_values(img1,img2,method,name,case,pair)
          original_values.append(1)
          save(conn, cursor,tuple(original_values))

          print('Phase two: Calculates the transformation')
          angles_mean = original_values[4]
          scale_mean = original_values[6]
          dst = gmt.affine_trans(img1,angles_mean,scale_mean)
          ploting_image_pair(dst,img2)
          _,_,_,values = prep_values(dst,img2,method,name,case,pair)
          values.append(2)

          save(conn, cursor,tuple(values))

          print("Phase three: Removes fake matches")
          angles_mean = original_values[4]
          angles_std = original_values[5]
          scale_mean = original_values[6]
          scale_std = original_values[7]

          angles_dif,scales = gmt.remove_fake_matches(matches,angles_dif,angles_mean,angles_std,scales,scale_mean,scale_std)

          angles_mean = stats.tstd(angles_dif)
          angles_std = stats.tstd(angles_dif)
          scale_mean = stats.tmean(scales)
          scale_std = stats.tstd(scales)

          dst = gmt.affine_trans(img1,angles_mean,scale_mean)
          ploting_image_pair(dst,img2)

          _,_,_,values = prep_values(dst,img2,method,name,case,pair)

          values.append(3)

          save(conn, cursor,tuple(values))

        del img1
        del img2
    conn.close()
    executeTimeF = time()
    print('Test executed in {} seconds'.format(executeTimeF-executeTimeI))

```

```python
if(__name__ == '__main__'):
    main()
```