# $\ell_0$-linear layer implemenation

Almost verbatim from [1712.01312](https://arxiv.org/abs/1712.01312.pdf)

Efficient gradient based optimization of the expected $\ell_0$ `norm` of parameter $\theta$. Let $s$ be a continuous rv with $q(s\mid \phi)$. The binary gates $z$ are driven by
$$
    z = \min\bigl\{\max\{s, 0\}1, \bigr\}
    \,, s \sim q(s\big\vert \phi)
    \,, $$
where the weights of a linear layer become $w = \theta \odot z$. This formulation allows the gate to be exactly zero, due to the underlying continueous rv $s$. Now
$$ \Pr\bigl(z\neq 0\big\vert \phi\bigr)
    = \int_0^{+\infty} q(s\big\vert \phi)ds
    \,. $$

The regularized ERM thus becomes
$$
    \mathbb{E}_{q(s\mid \phi)}
        \hat{\mathbb{E}}_{x,y \sim \mathcal{S}}
            \ell(h(x; \theta \odot g(s)), y)
        + \tfrac\lambda{\lvert \theta \rvert}
            \sum_j \Pr(z_j \neq 0\big\vert \phi_j)
    \,, $$
for $g\colon \mathbb{R} \to [0, 1] \colon x \mapsto \min\{1, \max\{x, 0\}\}$.

Let's use the reparametrization trick on $s$:
$$
    q(\cdot \vert\phi)
        \sim f(\phi, \varepsilon)
        \,, \varepsilon \sim p
    \,.$$
The hard-concrete distribution does this trick. [concrete.pdf](http://www.stats.ox.ac.uk/~cmaddis/pubs/concrete.pdf) Which is a concrete binary rv passed thorugh a hard-sigmoid.

So consider an rv $s\in (0, 1)$ with pdf $q_s(s\mid \phi)$ and cdf $Q_s(s\mid \phi)$. The rv is parameterized by $\log \alpha$ -- the location, and $\beta$ the temperature (recall the gumbel softmax trick -- it seems to be related). Let also $(\gamma, \zeta)$ be the parameters $\gamma < 0 < \zeta$, which strech the distribution of $s$ to $(\gamma, \zeta)$ interval.

$$
    u \sim \mathrm{U}(0, 1)
    \,, s = \sigma_{\tfrac1\beta}(\log \tfrac{u}{1-u} + \log \alpha)
    \,, \bar{s} = (\zeta - \gamma) s + \gamma
    \,, z = g(\bar{s})
    \,, $$
where $\sigma_\beta\colon x \mapsto \tfrac1{1+e^{-\beta x}}$, and $s$ is concrete rv, $g$ is the hard-sigmoid.

For $t\in (0, 1)$
$$
    \{\sigma_{\tfrac1\beta} \leq t\}
        = \{
            \tfrac{1-t}{t} \leq e^{-\tfrac1\beta x}
        \}
        = \{
            \log\tfrac{1-t}{t} \leq -\tfrac1\beta x
        \}
        = \{
            x \leq \beta\log\tfrac{t}{1-t}
        \}
    \,. $$

For any $u \in (0, 1)$
$$
    \{u\leq \sigma(x)\}
        = \{ 1+e^{-x} \leq \tfrac1u \}
        = \{ -x \leq \log\tfrac{1-u}u \}
        = \{ \log\tfrac{u}{1-u} \leq x \}
    \,. $$

Thus
$$\begin{align}
    \Pr\bigl(z = 0\big\vert \phi\bigr)
        &= \Pr\bigl(\bar{s} \leq 0 \big\vert \phi\bigr)
        = \Pr\bigl(s \leq \tfrac{-\gamma}{\zeta - \gamma} \big\vert \phi\bigr)
        \\
        &= \Pr\bigl(
            \log \tfrac{u}{1-u} + \log \alpha
                \leq \beta \log \tfrac{\tfrac{-\gamma}{\zeta - \gamma}}{1-\tfrac{-\gamma}{\zeta - \gamma}}
        \bigr)
        \\
        &= \Pr\bigl(
            \log \tfrac{u}{1-u} \leq \beta \log \tfrac{-\gamma}{\zeta} - \log \alpha
        \bigr)
        \\
        &= \Pr\Bigl(
            u \leq \sigma\bigl(\beta \log \tfrac{-\gamma}{\zeta} - \log \alpha\bigr)
        \Bigr)
        \,.
\end{align}$$

For $t \in (0, 1)$:
$$\begin{align}
    \Pr\bigl(z \leq t \big \vert \phi\bigr)
        &= \Pr\bigl( s \leq \tfrac{t - \gamma}{\zeta - \gamma} \big \vert \phi\bigr)
        = \Pr\bigl(
            \log \tfrac{u}{1-u} + \log \alpha
                \leq \beta \log \tfrac{\tfrac{t - \gamma}{\zeta - \gamma}}{1-\tfrac{t - \gamma}{\zeta - \gamma}}
        \bigr)
        \\
        &= \Pr\bigl(
            \log \tfrac{u}{1-u}
                \leq \beta \log \tfrac{t - \gamma}{\zeta - t}
                    - \log \alpha
        \bigr)
        = \sigma\bigl(
            \beta \log \tfrac{t - \gamma}{\zeta - t} - \log \alpha
        \bigr)
        \,.
\end{align}
$$

Hence
$$\begin{align}
    \Pr\bigl(z \neq 0\big\vert \phi\bigr)
        &= 1 - \Pr\Bigl(
            u \leq \sigma\bigl(\beta \log \tfrac{-\gamma}{\zeta} - \log \alpha\bigr)
        \Bigr)
        \\
        &= \Pr\Bigl(
            u \geq \sigma\bigl(\beta \log \tfrac{-\gamma}{\zeta} - \log \alpha\bigr)
        \Bigr)
        \\
        &= \Pr\Bigl(
            1-u \leq 1-\sigma\bigl(\beta \log \tfrac{-\gamma}{\zeta} - \log \alpha\bigr)
        \Bigr)
        \\
        &= \Pr\Bigl(
            1-u \leq \sigma\bigl(\log \alpha - \beta \log \tfrac{-\gamma}{\zeta}\bigr)
        \Bigr)
        \\
        &= \Pr\Bigl(
            u \leq \sigma\bigl(\log \alpha - \beta \log \tfrac{-\gamma}{\zeta}\bigr)
        \Bigr)
        \\
        &= \sigma\bigl(\log \alpha - \beta \log \tfrac{-\gamma}{\zeta}\bigr)
        \,.
\end{align}$$

Let's carefully write the distirbution of $z$:
$$\begin{align}
    \Pr\bigl(z \leq t\bigr)
        &= 0
        \,, \text{ for } t < 0
        \,, \\
    \Pr\bigl(z \leq t\bigr)
        &= \sigma\bigl(
            \beta \log \tfrac{t - \gamma}{\zeta - t} - \log \alpha
        \bigr)
        \,, \text{ for } t \in [0, 1)
        \,, \\
    \Pr\bigl(z \leq t\bigr)
        &= 1
        \,, \text{ for } t \geq 1
        \,.
\end{align}$$
This distribution has atoms at $0$ and $1$:
$$\begin{align}
    \Pr\bigl(z = 0\bigr)
        &= \Pr\bigl(z \leq 0\bigr) - \Pr\bigl(z < 0\bigr)
        = \sigma\bigl(
            \beta \log \tfrac{- \gamma}\zeta - \log \alpha
        \bigr) - \lim_{t\uparrow 0} \Pr\bigl(z \leq t\bigr)
        = \sigma\bigl(
            \beta \log \tfrac{- \gamma}\zeta - \log \alpha
        \bigr)
        \,, \\
    \Pr\bigl(z = 1\bigr)
        &= 1 - \lim_{t\uparrow 1} \Pr\bigl(z \leq t\bigr)
        = 1 - \sigma\bigl(
            \beta \log \tfrac{1 - \gamma}{\zeta - 1} - \log \alpha
        \bigr)
        = \sigma\bigl(
            \log \alpha - \beta \log \tfrac{1 - \gamma}{\zeta - 1}
        \bigr)
        \,.
\end{align}$$

For $t\in (0, 1)$ the pdf of $z$ is
$$
    \tfrac{d}{dt} \Pr(z \leq t)
        = \tfrac{d}{dt} \sigma\bigl(
            \beta \log \tfrac{t - \gamma}{\zeta - t} - \log \alpha
        \bigr)
        = \sigma(x) \sigma(-x) \big\vert_{x = \beta \log \tfrac{t - \gamma}{\zeta - t} - \log \alpha}
%         \Bigl( \beta \tfrac1{t - \gamma} + \beta \tfrac{1}{\zeta - t} \Bigr)
%         = (\ldots)
        \beta \tfrac{\zeta - \gamma}{(t - \gamma)(\zeta - t)}
%         \beta (\log{t - \gamma} - \log {\zeta - t})
%         \tfrac{- e^{-x}}{(1+e^{-x})^2}
    \,. $$

The expectation of $z$:
$$
\begin{align}
    \mathbb{E}_{\mathrm{hc}} z
        &= \int z d\mathbb{P}
        = \int_{\{0\}} + \int_{(0, 1)} + \int_{\{1\}}
        = 0 \sigma\bigl(
            \beta \log \tfrac{- \gamma}\zeta - \log \alpha
        \bigr)
        + 1 \sigma\bigl(
            \log \alpha - \beta \log \tfrac{1 - \gamma}{\zeta - 1}
        \bigr)
        + \int_{(0, 1)}
            t \tfrac{d}{dt} \Pr(z \leq t)
        dt
        \\
        &= \sigma\bigl(
            \log \alpha - \beta \log \tfrac{1 - \gamma}{\zeta - 1}
        \bigr)
        + \sigma\bigl(
                \beta \log \tfrac{1 - \gamma}{\zeta - 1} - \log \alpha
            \bigr)
        - \int_{(0, 1)}
            \sigma\bigl(
                \beta \log \tfrac{t - \gamma}{\zeta - t} - \log \alpha
            \bigr) dt
        \\
        &= 1 - \int_{(0, 1)}
            \sigma\bigl(
                \beta \log \tfrac{t - \gamma}{\zeta - t} - \log \alpha
            \bigr) dt
%         = \int_0^1 \Pr(z \geq t) dt
    \,.
\end{align}$$

Consider a linear transformtaion $y = Wx + b$ for $ W = \theta \odot z$, for $x\in \mathbb{R}^m$ and $y\in \mathbb{R}^n$. Then
$$\begin{align}
    y_i &= b_i + \sum_j \theta_{ij} z_{ij} x_j
        = b_i + \sum_j z_{ij} e_i^\top \theta e_j x_j
        \\
        &= b_i + e_i^\top \mathop{diag}(z_i) \theta x
    \,.
\end{align}
$$
Local reparametrization trick? The ICLR2018 paper uses a single sample (!) per minibatch. For otherwise we need `batch x n x m` samples, which is a lot.

$$\begin{align}
    \partial y
        &= \partial W x + W \partial x
        = \bigl( \partial \theta \odot z + \theta \odot \partial z \bigr) x + W \partial x
        \\
        &= \bigl( \partial \theta \odot z + \theta \odot \partial z \bigr) x + W \partial x
        \,.
\end{align}$$

See the last paragraph before section 3 of their paper! There they say that they use group dropout.

Linear layer $\mathbb{R}^n \to \mathbb{R}^m$
$$
    y = W x + b
    \,, W_{ij} = \theta_{ij} z_{ij}
\,, $$

where $z \in [0, 1]^{n\times m}$ is a learnable variational dropout mask

$$
    u_{ij} \sim \mathrm{U}(0, 1)
    \,, s_{ij} = \sigma\bigl(\tfrac1\beta (\log \tfrac{u_{ij}}{1-u_{ij}} + \log \alpha_{ij})\bigr)
    \,, \\
    z_{ij} = \min\bigl\{1, \max\{0, (\zeta - \gamma) s_{ij} + \gamma\}\bigr\}
    \,, \sigma\colon x \mapsto \tfrac1{1+e^{-x}}
    \,.
$$

$\gamma=-0.1, \zeta=+1.1$, $\beta = 0.66$.

Objective
$$
    \tfrac1{2B} \sum_b \|g(x_b) - y_b \|_2^2
        + \lambda \tfrac1{\# \text{par}}
            \sum_{ij} \Pr(z_{ij} \neq 0)
    \,. $$

The $\mathbb{K}$-brick (block) ($\mathbb{K} = \mathbb{R}$ or $\mathbb{C}$) has the following action:

$$
    F_l
    \colon \mathbb{K}^{n_{l-1}} \to \mathbb{K}^{n_l}
    \colon z \mapsto P_l \biggl(
        \Bigl(
            g_{lj}(
                \lvert z \rvert^{\color{red} 2}
            )
            \mathbf{\color{red} \odot}
        z \Bigr)_{j=0}^{C_l}
    \biggr)
    \,, $$

with $
g_{lj} \colon
    \mathbb{R}^{n_{l-1}} \to \mathbb{K}^{n_{l-1}}
$ are the gains
of each one of $C_l$ channels of the $l$-th "hump". By design $
g_{l0}(\cdot)
    = \mathbf{1} \in \mathbb{K}^{n_{l-1}}
$, and a projection
$
P_l \colon
    \mathbb{K}^{(1 + C_l) \times n_{l-1}} \to \mathbb{K}^{n_l}
$.

Maps $x_0 \mapsto y(x_0)$ with $
x_l = F_l(x_{l-1}) + x_0
$, $l=1..L$

and $y = P_\infty(x_L)$

The network maps $
x_l
    = x_{l-1} + z_l (h_l - x_{l-1})
$ (bricks $l=1..L$) and $y = P_\infty(x_L)$

* (**baseline**) $
h_l = F_l(x_{l-1}) + x_0
$,

* (**residual**) $
h_l = F_l(x_{l-1})
$

where $z_l \in [0, 1]$ is the utilization mask (random from Var-$\ell_0$).

$z_j \sim q(z\vert \alpha_j(x_j))$:

$$
    u_j \sim \mathrm{U}(0, 1)
    \,, s_j = \sigma\bigl(\tfrac1\beta (\log \tfrac{u_j}{1-u_j} - \alpha_j(x_j))\bigr)
    \,, \\
    z_j = \min\bigl\{1, \max\{0, (\zeta - \gamma) s_j + \gamma\}\bigr\}
    \,, \sigma\colon p \mapsto \tfrac1{1+e^{-p}}
    \,,
$$

where $\gamma=-0.1, \zeta=+1.1$, $\beta = 0.66$, and $\alpha_j(\cdot)$ is the gating `controller`
of brick $j$.

$z_j$ has non-negligible atoms at $0$ and $1$ and
it's distribution is differentiable

$$
\mathrm{mse}
    + \tfrac{C}{\lvert B\rvert} \sum_{i\in B}
        \overbrace{
            \sum_j \mathbb{P}(z_j \neq 0\lvert x_i)
        }^{\text{utilization}}
\,. $$

```python
import torch
import torch.nn.functional as F

import numpy as np

%matplotlib inline
import matplotlib.pyplot as plt

from cplxmodule.relevance import named_penalties, penalties

from torch.nn import Linear
from cplxmodule.relevance import LinearARD
from cplxmodule.relevance import LinearL0ARD, LinearLASSO
```

```python
shape = 250, 50
```

```python
index = np.r_ if True else np.random.permutation(shape[0])
index = index[:shape[1]]
```

```python
model_ard = LinearL0ARD(*shape, bias=False, reduction="mean", group=None)  # longer to learn on small data, noisy grads indeed
# model_ard = LinearARD(*shape, bias=False)
# model_ard = Linear(*shape, bias=False)

# model_ard = LinearLASSO(*shape, bias=False)
```

```python
from cplxmodule.masked import LinearMasked

model_masked = LinearMasked(*shape, model_ard.bias is not None)
```

Train on a simple identiy mapping

```python
model = model_masked
```

```python
model = model_ard
```

```python
import tqdm

optim = torch.optim.Adam(model.parameters())
losses = []

model.train()
x = torch.randn(100, shape[0])
y = -x[:, index]

with tqdm.tqdm(range(8000)) as bar:
    for epoch in bar:
        optim.zero_grad()

        loss = F.mse_loss(model(x), y)
        loss += sum(penalties(model))

        loss.backward()

        optim.step()
        losses.append(float(loss))
    # end for
# end with

plt.semilogy(losses)
```

```python
sum(penalties(model))
```

```python
from cplxmodule.relevance import sparsity

import math

tau = 0.5

sparsity(model, threshold=tau)
```

```python
from dpd.tools.utils import sparsity_details
```

```python
[*sparsity_details(model, None)]
```

```python
model.eval()

test_loss = []
for _ in tqdm.tqdm(range(1000)):
    x = torch.randn(100, shape[0])
    y = -x[:, index]

    test_loss.append(float(F.mse_loss(model(x), y)))

plt.hist(test_loss, bins=51);
```

```python
from cplxmodule.masked import is_sparse

fig, ax = plt.subplots(3, 1, figsize=(12, 6))

with torch.no_grad():
    if "LinearL0ARD" in globals() and isinstance(model, LinearL0ARD):
        z = model.gate(None).detach()
        ax[0].imshow(z.numpy(), cmap=plt.cm.bone)
        ax[0].set_title(r"Gate $z_{ij}$")

    elif is_sparse(model):
        z = model.mask
        ax[0].imshow(z.numpy(), cmap=plt.cm.bone)
        ax[0].set_title(r"Gate $z_{ij}$")

    ax[1].imshow(abs(getattr(model, "weight_masked", model.weight)).numpy(), cmap=plt.cm.binary_r)
    ax[1].set_title(r"Absolute learnt weight $| \theta_{ij} |$")

    if isinstance(model, LinearMasked):
        relevance = 1 - model.mask
    else:
        relevance = model.log_alpha
    ax[2].imshow(relevance.detach().numpy(), cmap=plt.cm.bone_r)
    ax[2].set_title(r"Relevance $\log \alpha_{ij}$ / mask")

plt.tight_layout()
plt.show()
```

```python
model.weight[np.r_[:shape[1]], index]
```

```python
from cplxmodule.masked import compute_ard_masks, deploy_masks

masks = compute_ard_masks(model_ard, threshold=0)

model_masked = deploy_masks(model_masked, state_dict=masks)
```

```python
[*model_masked.named_buffers()]
```

```python
model_masked.weight_masked
```

```python
# [*model.named_parameters()]
```

```python
model_masked.state_dict()
```

```python
assert False
```

```python
from cplxmodule.relevance.base import named_relevance, named_sparsity
```

```python
from dpd.models import ModelHUAWEI
from dpd.models.huawei import ModelHUAWEIBoosted
```

```python
recipe = ModelHUAWEI.default_recipe(32, 4)
```

```python
# model = ModelHUAWEI(*recipe, n_columns=1, ard=True)
model = ModelHUAWEIBoosted(*recipe, n_columns=1, linear=LinearARD)
```

```python
self = model["boost00"][0]["dense00"]
```

```python

```

```python
[*sparsity_details(model, 4.0)]
```

```python
import math
import torch
import torch.nn.functional as F
from torch.nn import Parameter

%matplotlib inline
import matplotlib.pyplot as plt

from cplxmodule.relevance import named_penalties, penalties
from cplxmodule.relevance.base import BaseARD
```

```python
class LinearL0ARD(torch.nn.Linear, BaseARD):
    """L0 regularized linear layer according to [1]_.

    Details
    -------
    IMPORTANT: This implemetnation use -ve log-alpha parametrization
    in order to keep the layer's parameters interpretation aligned
    with the interpretation in variational dropout layer of Kingma
    et al. (relevance.LinearARD).

    References
    ----------
    [1]_ :: https://arxiv.org/abs/1712.01312.pdf
    [2]_ :: https://arxiv.org/abs/1902.09574.pdf
    """
    beta, gamma, zeta = .25, -0.1, 1.1
    def __init__(self, in_features, out_features, bias=True):
        super().__init__(in_features, out_features, bias=bias)

        self.log_alpha = Parameter(
            torch.Tensor(*self.weight.shape))
        self.reset_variational_parameters()

    def reset_variational_parameters(self):
        # assume everything is important (but do not saturate the sigmoid too much)
        self.log_alpha.data.uniform_(-0.45, -0.45)

    def forward(self, input):
        if not self.training:
            # suppose u = 0.5, the the input is zero
            z = self.sample_hc(0.0, beta=self.beta)  # 1.0 in paper eq(13), self.beta in code
            return F.linear(input, self.weight * z, self.bias)

        # a single mask sample for the whole batch!
        u = torch.rand_like(self.log_alpha)
        z = self.sample_hc(torch.log(u) - torch.log(1 - u), beta=self.beta)
        return F.linear(input, self.weight * z, self.bias)

    def sample_hc(self, logit=0.0, beta=1.0):
        s = torch.sigmoid((logit - self.log_alpha) / beta)
        return torch.clamp((self.zeta - self.gamma) * s + self.gamma, 0, 1)

    @property
    def penalty(self):
        shift = self.beta * math.log(- self.gamma / self.zeta)
        return 1 - torch.sigmoid(self.log_alpha + shift).mean()

    def get_sparsity_mask(self, threshold):
        r"""Get the dropout mask based on the log-relevance."""
        with torch.no_grad():
            return torch.ge(self.log_alpha, threshold)
```

```python
class LinearL0_arxiv171201312(torch.nn.Linear, BaseARD):
    """L0 regularized linear layer according to [1]_."""
    beta, gamma, zeta = .5, -0.1, 1.1

    def __init__(self, in_features, out_features, bias=True):
        super().__init__(in_features, out_features, bias=bias)

        self.log_alpha = Parameter(torch.Tensor(*self.weight.shape))
        self.reset_variational_parameters()

    def reset_variational_parameters(self):
        # 0.6 = a / (a+1) = n / (n + m), for a = n/m, n=2, m=3. put n, m = 9, 11
        self.log_alpha.data.normal_(0.45, std=0.01)

    def forward(self, input):
        if not self.training:
            s = torch.sigmoid(self.log_alpha)
            z = torch.clamp((self.zeta - self.gamma) * s + self.gamma, 0, 1)
            return F.linear(input, self.weight * z, self.bias)

        # a single mask sample for the whole batch!
        u = torch.rand_like(self.log_alpha)
        logit = torch.log(u) - torch.log(1 - u) + self.log_alpha
        s = torch.sigmoid(logit / self.beta)
        z = torch.clamp((self.zeta - self.gamma) * s + self.gamma, 0, 1)
        return F.linear(input, self.weight * z, self.bias)

    @property
    def penalty(self):
        shift = self.beta * math.log(- self.gamma / self.zeta)
        return torch.sigmoid(self.log_alpha - shift).mean()
```

```python
x = torch.randn(1000, 50) * 10
x1 = F.hardtanh(x, 0., 1.)
x2 = torch.clamp(x, 0., 1.)

assert torch.allclose(x2, x1)
assert torch.allclose(x1, x2)
```

```python
from cplxmodule.relevance.base import BaseARD
from cplxmodule.relevance.utils import torch_sparse_linear, torch_sparse_tensor
from cplxmodule.relevance.utils import parameter_to_buffer, buffer_to_parameter

from torch.nn import Parameter

class Dummy(torch.nn.Linear):
    @property
    def is_sparse(self):
        mode = getattr(self, "sparsity_mode_", None)
        return mode is not None

    def forward(self, input):
        if self.is_sparse:
            return self.forward_sparse(input)
        return super().forward(input)

    def forward_sparse(self, input):
        if self.sparsity_mode_ == "dense":
            weight = self.weight_ * self.nonzero_
            return F.linear(input, weight, self.bias)

        else:
            weight = torch_sparse_tensor(self.nonzero_, self.weight_,
                                         self.weight.shape)
            return torch_sparse_linear(input, weight, self.bias)

    def sparsify(self, threshold=1.0, mode="dense"):
        if mode is not None and mode not in ("dense", "sparse"):
            raise ValueError(f"""`mode` must be either 'dense', 'sparse' or """
                             f"""`None` (got '{mode}').""")

        if mode is not None:
            with torch.no_grad():
                mask = torch.gt(abs(self.weight), threshold)

            if mode == "sparse":
                # truly sparse mode
                weight = self.weight.data[mask].clone()
                self.register_buffer("nonzero_", mask.nonzero().t())

            elif mode == "dense":
                # smiluated sparse mode
                mask = mask.to(self.weight).data
                weight = self.weight.data * mask
                self.register_buffer("nonzero_", mask)

            # make weight into a buffer (load_state dict doesn't care
            #  about param/buffer distinction!)
            self.register_parameter("weight_", torch.nn.Parameter(weight))
            parameter_to_buffer(self, "weight")

        elif self.is_sparse:
            # reinstate the weight as the parameter and delete runtime stuff
            del self.nonzero_, self.weight_
            buffer_to_parameter(self, "weight")

        # end if

        self.sparsity_mode_ = mode

        return self
```

```python
from cplxmodule.layers import CplxParameter, CplxLinear
from cplxmodule import Cplx
```

```python
def parameter_to_buffer(module, name):
    # par could be a solo parameter or a container (essentially a submodule)
    par = getattr(module, name)
    if isinstance(par, (torch.nn.ParameterDict, torch.nn.ParameterList)):
        # parameter containers no not use buffers and aren't expected to.
        #  So we hide parameters there. This precludes acces via __getitem__
        #  though. Not via __getattr__

        # create a copy of the container's master parameter dict's keys and mutate
        for name in list(par._parameters):
            # By design of Parameter containers this never recurses deeper
            parameter_to_buffer(par, name)
        return

    # a solo parameter
    if par is not None and not isinstance(par, torch.nn.Parameter):
        raise KeyError(f"parameter '{name}' is not a tensor.")

    # remove the parameter and mutate into a grad-detached buffer
    delattr(module, name)
    par = par.detach() if par is not None else None
    module.register_buffer(name, par)

def buffer_to_parameter(module, name):
    # a buffer here can be a buffer or a former mutated parameter container
    buf = getattr(module, name)
    if isinstance(buf, (torch.nn.ParameterDict, torch.nn.ParameterList)):
        # create a copy of the container's master buffer dict's keys and restore
        for name in list(buf._buffers):
            # By design of Parameter containers this never goes deeper
            #  than this call
            buffer_to_parameter(buf, name)
        return

    if buf is not None and not isinstance(buf, torch.Tensor):
        raise KeyError(f"buffer '{name}' is not a tensor.")

    # remove the buffer and mutate back into a proper parameter
    delattr(module, name)
    buf = torch.nn.Parameter(buf) if buf is not None else None
    module.register_parameter(name, buf)
```

```python
mod = torch.nn.Module()
mod.par = torch.nn.Parameter(torch.randn(10))
mod.par_d = torch.nn.ParameterDict({
    "par_1": torch.nn.Parameter(torch.randn(10)),
    "par_2": torch.nn.Parameter(torch.randn(10)),
})
mod.par_l = torch.nn.ParameterList([
    torch.nn.Parameter(torch.randn(10)),
    torch.nn.Parameter(torch.randn(10)),
])
```

```python
lin = CplxLinear(10, 10)

lin.test = torch.nn.ParameterList([
    torch.nn.Parameter(lin.weight.real.clone()),
    None,
    torch.nn.Parameter(lin.weight.real.clone())
])

lin.test_2 = torch.nn.ParameterDict({
    "a": torch.nn.Parameter(torch.randn(10)),
    "z": torch.nn.Parameter(torch.randn(10)),
    "_12omega": None,
})
```

```python
before = Cplx(**lin.weight)

parameter_to_buffer(lin, "weight")
print([n for n, b in lin.named_buffers()], list(lin.weight._buffers))

buffer_to_parameter(lin, "weight")
print([n for n, b in lin.named_buffers()], list(lin.weight._buffers))

assert np.allclose(Cplx(**lin.weight).detach().numpy(), before.detach().numpy())
```

```python
copy = [*lin.test]

parameter_to_buffer(lin, "test")
print([n for n, b in lin.named_buffers()], list(lin.test._buffers))

buffer_to_parameter(lin, "test")
print([n for n, b in lin.named_buffers()], list(lin.test._buffers))

assert all(a is None and b is None or torch.allclose(a, b)
           for a, b in zip(lin.test, copy))
```

```python
copy = {**lin.test_2}

parameter_to_buffer(lin, "test_2")
print([n for n, b in lin.named_buffers()], list(lin.test_2._buffers))

buffer_to_parameter(lin, "test_2")
print([n for n, b in lin.named_buffers()], list(lin.test_2._buffers))

assert all(a is None and b is None or torch.allclose(a, b)
           for a, b in zip(lin.test_2.values(), copy.values()))
```

CDF

```python
import math
from scipy.special import expit

beta, gamma, zeta = 0.95, -0.5, 1.5
```

```python
mesh = np.meshgrid(np.linspace(-.25, 1.25, num=501),
                   np.linspace(-12, 12, num=101))

t, log_a = mesh

tcl = np.clip(t, 0, 1)
cdf = expit(beta * (np.log(tcl - gamma) - np.log(zeta - tcl)) - log_a)
cdf[t >= 1.] = 1.
cdf[t < 0] = 0.

from mpl_toolkits.mplot3d import Axes3D
fig = plt.figure(figsize=(12, 12))
ax = fig.add_subplot(111, projection='3d')

ax.plot_surface(t, log_a, cdf)

ax.view_init(35, 120)
plt.show()
```

Slices of CDF at large $\log \alpha$

```python
log_a[-50:-35, 0]
```

```python
t[0][-90:]
```

```python
# plt.plot(t[0], np.diff(cdf, axis=1, prepend=0).T)
plt.plot(t[0], cdf[-50:-35].T)
plt.show()
```

```python
# plt.plot(t[0], np.diff(cdf, axis=1, prepend=0).T)
plt.plot(log_a.T[0], cdf.T[-90:-84].T)
plt.show()
```

$p \mapsto \log \alpha$ such that
$$
    p = \Pr(z=1)
        = \sigma\bigl( \log \alpha - \beta \log\tfrac{1-\gamma}{\zeta- 1}\bigr)
    \,.
$$

```python
from scipy.special import logit

eps = 1
beta, gamma, zeta = 0.5, -0-eps, 1.+eps

p = np.linspace(0.0, 1.0, num=101)[1:-1]
plt.plot(p, logit(p) + beta * (math.log(1 - gamma) - math.log(zeta - 1)))
```

```python
0.5, -0.15, 1.15
```

```python
beta, zeta, gamma = .1, 1.25, -0.25

la = np.linspace(-8, 8, num=1001)
```

```python
u = np.random.rand(1001, 10000)
z = np.log(u) - np.log(1-u)
```

```python
mc = expit((z-la[:, np.newaxis])/beta).mean(axis=-1)
pp = expit(-la)
```

```python
plt.plot(mc-pp)
```


# QUANTUM PHASE ESTIMATION

This tutorial provides a detailed implementation of the Quantum Phase Estimation (QPE) algorithm using the Amazon Braket SDK.
The QPE algorithm is designed to estimate the eigenvalues of a unitary operator $U$ [1, 2];
it is a very important subroutine to many quantum algorithms, most famously Shor's algorithm for factoring and the HHL algorithm (named after the physicists Harrow, Hassidim and Lloyd) for solving linear systems of equations on a quantum computer [1, 2].
Moreover, eigenvalue problems can be found across many disciplines and application areas, including (for example) principal component analysis (PCA) as used in machine learning or the solution of differential equations as relevant across mathematics, physics, engineering and chemistry.
We first review the basics of the QPE algorithm.
We then implement the QPE algorithm in code using the Amazon Braket SDK, and we illustrate the application thereof with simple examples.
This notebook also showcases the Amazon Braket `circuit.subroutine` functionality, which allows us to use custom-built gates as if they were any other built-in gates.
This tutorial is set up to run either on the local simulator or the managed simulators; changing between these devices merely requires changing one line of code as demonstrated as follows in cell [4].

## TECHNICAL BACKGROUND OF QPE

__Introduction__: A unitary matrix is a complex, square matrix whose adjoint (or conjugate transpose) is equal to its inverse. Unitary matrices have many nice properties, including the fact that their eigenvalues are always roots of unity (that is, phases). Given a unitary matrix $U$ (satisfying $U^{\dagger}U=\mathbb{1}=UU^{\dagger}$) and an eigenstate $|\psi \rangle$ with $U|\psi \rangle = e^{2\pi i\varphi}|\psi \rangle$, the Quantum Phase Estimation (QPE) algorithm provides an estimate $\tilde{\varphi} \approx \varphi$ for the phase $\varphi$ (with $\varphi \in [0,1]$ since the eigenvalues $\lambda = \exp(2\pi i\varphi)$ of a unitary have modulus one).
The QPE works with high probability within an additive error $\varepsilon$ using $O(\log(1/\varepsilon))$ qubits (without counting the qubits used to encode the eigenstate) and $O(1/\varepsilon)$ controlled-$U$ operations [1].

__Quantum Phase Estimation Algorithm__:
The QPE algorithm takes a unitary $U$ as input. For the sake of simplicity (we will generalize the discussion below), suppose that the algorithm also takes as input an eigenstate $|\psi \rangle$ fulfilling

$$U|\psi \rangle = \lambda |\psi \rangle,$$

with $\lambda = \exp(2\pi i\varphi)$.

QPE uses two registers of qubits: we refer to the first register as *precision* qubits (as the number of qubits $n$ in the first register sets the achievable precision of our results) and the second register as *query* qubits (as the second register hosts the eigenstate $|\psi \rangle$).
Suppose we have prepared this second register in $|\psi \rangle$.  We then prepare a uniform superposition of all basis vectors in the first register using a series of Hadamard gates.

Next, we apply a series of controlled-unitaries $C-U^{2^{k}}$ for different powers of $k=0,1,\dots, n-1$ (as illustrated in the circuit diagram that follows).
For example, for $k=1$ we get
\begin{equation}
\begin{split}
(|0 \rangle + |1 \rangle) |\psi \rangle  & \rightarrow |0 \rangle |\psi \rangle + |1 \rangle U|\psi \rangle \\
& = (|0 \rangle + e^{2\pi i \varphi}|1 \rangle) |\psi \rangle.
\end{split}
\end{equation}

Note that the second register remains unaffected as it stays in the eigenstate $|\psi \rangle$.
However, we managed to transfer information about the phase of the eigenvalue of $U$ (that is, $\varphi$) into the first *precision* register by encoding it as a relative phase in the state of the qubits in the first register.

Similarly, for $k=2$ we obtain
\begin{equation}
\begin{split}
(|0 \rangle + |1 \rangle) |\psi \rangle  & \rightarrow |0 \rangle |\psi \rangle + |1 \rangle U^{2}|\psi \rangle \\
& = (|0 \rangle + e^{2\pi i 2\varphi}|1 \rangle) |\psi \rangle,
\end{split}
\end{equation}

where this time we wrote $2\varphi$ into the precision register. The process is similar for all $k>2$.

Introducing the following notation for binary fractions
$$[0. \varphi_{l}\varphi_{l+1}\dots \varphi_{m}] = \frac{\varphi_{l}}{2^{1}} + \frac{\varphi_{l+1}}{2^{2}} + \frac{\varphi_{m}}{2^{m-l+1}},$$

one can show that the application of a controlled unitary $C-U^{2^{k}}$ leads to the following transformation

\begin{equation}
\begin{split}
(|0 \rangle + |1 \rangle) |\psi \rangle  & \rightarrow |0 \rangle |\psi \rangle + |1 \rangle U^{2^{k}}|\psi \rangle \\
& = (|0 \rangle + e^{2\pi i 2^{k}\varphi}|1 \rangle) |\psi \rangle \\
& = (|0 \rangle + e^{2\pi i [0.\varphi_{k+1}\dots \varphi_{n}]}|1 \rangle) |\psi \rangle,
\end{split}
\end{equation}

where the first $k$ bits of precision in the binary expansion (that is, those bits to the left of the decimal) can be dropped, because $e^{2\pi i \theta} = 1$ for any whole number $\theta$.

The QPE algorithm implements a series of these transformations for $k=0, 1, \dots, n-1$, using $n$ qubits in the precision register.
In its entirety, this sequence of controlled unitaries leads to the transformation

$$ |0, \dots, 0 \rangle \otimes |\psi \rangle \longrightarrow
(|0 \rangle + e^{2\pi i [0.\varphi_{n}]}|1 \rangle)
\otimes (|0 \rangle + e^{2\pi i [0.\varphi_{n-1}\varphi_{n}]}|1 \rangle)
\otimes \dots
\otimes (|0 \rangle + e^{2\pi i [0.\varphi_{1}\dots\varphi_{n}]}|1 \rangle)
\otimes |\psi \rangle.
$$

By inspection, one can see that the state of the register qubits above corresponds to a quantum Fourier transform of the state $|\varphi_1,\dots,\varphi_n\rangle$. Thus, the final step of the QPE algorithm is to run the *inverse* Quantum Fourier Transform (QFT) algorithm on the precision register to extract the phase information from this state. The resulting state is
$$|\varphi_{1}, \varphi_{2}, \dots, \varphi_{n}  \rangle \otimes |\psi\rangle.$$

Measuring the precision qubits in the computational basis then gives the classical bitstring $\varphi_{1}, \varphi_{2}, \dots, \varphi_{n}$, from which we can readily infer the phase estimate $\tilde{\varphi} = 0.\varphi_{1} \dots \varphi_{n}$ with the corresponding eigenvalue $\tilde{\lambda} = \exp(2\pi i \tilde{\varphi})$.

__Simple example for illustration__: For concreteness, consider a simple example with the unitary given by the Pauli $X$ gate, $U=X$, for which $|\Psi \rangle = |+\rangle = (|0 \rangle + |1 \rangle)/\sqrt{2}$ is an eigenstate with eigenvalue $\lambda = 1$, i.e., $\varphi=0$.
This state can be prepared with a Hadamard gate as $|\Psi \rangle = H|0 \rangle$.
We take a precision register consisting of just two qubits ($n=2$).

Thus, after the first layer of Hadamard gates, the quantum state is
$$|0,0,0 \rangle \rightarrow |+,+,+\rangle.$$

Next, the applications of the controlled-$U$ gates (equal to $C-X$ operations, or CNOT gates in this example) leave this state untouched, because $|+\rangle$ is an eigenstate of $X$ with eigenvalue $+1$.
Finally, applying the inverse QFT leads to

$$\mathrm{QFT}^{\dagger}|+++\rangle=\mathrm{QFT}^\dagger\frac{|00\rangle + |01\rangle + |10\rangle + |11\rangle}{4}\otimes |+\rangle = |00\rangle \otimes |+\rangle,$$

from which we deduce $\varphi = [0.00]=0$ and therefore $\lambda=1$, as expected.
Here, in the last step we have used $|00\rangle + |01\rangle + |10\rangle + |11\rangle = (|0\rangle + e^{2\pi i[0.0]}|1\rangle)(|0\rangle + e^{2\pi i[0.00]}|1\rangle)$, which makes the effect of the inverse QFT more apparent.

__Initial state of query register__: So far, we have assumed that the query register is prepared in an eigenstate $|\Psi\rangle$ of $U$. What happens if this is not the case? Let's reconsider the simple example given previously.

Suppose now that the query register is instead prepared in the state $|\Psi\rangle = |1\rangle$.
We can always express this state in the eigenbasis of $U$, that is, $|1\rangle = \frac{1}{\sqrt{2}}(|+\rangle - |-\rangle)$.
By linearity, application of the QPE algorithm then gives (up to normalization)

\begin{equation}
\begin{split}
\mathrm{QPE}(|0,0,\dots\rangle \otimes |1\rangle) & = \mathrm{QPE}(|0,0,\dots\rangle \otimes |+\rangle)
- \mathrm{QPE}(|0,0,\dots\rangle \otimes |-\rangle) \\
& =  |\varphi_{+}\rangle \otimes |+\rangle - |\varphi_{-}\rangle \otimes |-\rangle. \\
\end{split}
\end{equation}

When we measure the precision qubits in this state, 50% of the time we will observe the eigenphase $\varphi_{+}$ and 50% of the time we will measure $\varphi_{-}$. We illustrate this example numerically as follows.

This example motivates the general case: we can pass a state that is not an eigenstate of $U$ to the QPE algorithm, but we may need to repeat our measurements several times in order to obtain an estimate of the desired phase.

## CIRCUIT IMPLEMENTATION OF QPE

The QPE circuit can be implemented using Hadamard gates, controlled-$U$ unitaries, and the inverse QFT (denoted as $\mathrm{QFT}^{-1}$).
The details of the calculation can be found in a number of resources (such as, [1]); we omit them here.
Following the previous discussion, the circuit that implements the QPE algorithm reads as below, where m is the size of lower query register and n is the size of upper precision register.

## IMPORTS and SETUP

```python
# general imports
import numpy as np
import math
import matplotlib.pyplot as plt
# magic word for producing visualizations in notebook
%matplotlib inline
```

```python
# AWS imports: Import Amazon Braket SDK modules
from braket.circuits import Circuit, circuit
from braket.devices import LocalSimulator
from braket.aws import AwsDevice
```

```python
# local imports
from utils_qpe import qpe, run_qpe

%load_ext autoreload
%autoreload 2
```

```python
# set up device: local simulator or the managed cloud-based simulator
# device = LocalSimulator()
device = AwsDevice("arn:aws:braket:::device/quantum-simulator/amazon/sv1")
```

### Pauli Matrices:
In some of our examples, we choose the unitary $U$ to be given by the **Pauli Matrices**, which we thus define as follows:

```python
# Define Pauli matrices
Id = np.eye(2)             # Identity matrix
X = np.array([[0., 1.],
              [1., 0.]])   # Pauli X
Y = np.array([[0., -1.j],
              [1.j, 0.]])  # Pauli Y
Z = np.array([[1., 0.],
              [0., -1.]])  # Pauli Z
```

## IMPLEMENTATION OF THE QPE CIRCUIT

In ```utils_qpe.py``` we provide simple helper functions to implement the quantum circuit for the QPE algorithm.
Specifically, we demonstrate that such modular building blocks can be registered as subroutines, using ```@circuit.subroutine(register=True)```.
Moreover, we provide a helper function (called ```get_qpe_phases```) to perform postprocessing based on the measurement results to extract the phase. The details of ```utils_qpe.py``` are shown in the Appendix.

To implement the unitary $C-U^{2^k}$, one can use the fact that $C-U^{2} = (C-U)(C-U)$, so that $C-U^{2^{k}}$ can be constructed by repeatedly applying the core building block $C-U$.
However, the circuit generated using this approach will have a significantly larger depth. In our implementation, we instead define the matrix $U^{2^k}$ and create the controlled $C-(U^{2^k})$ gate from that.

## VISUALIZATION OF THE QFT CIRCUIT

To check our implementation of the QPE circuit, we visualize this circuit for a small number of qubits.

```python
# set total number of qubits
precision_qubits = [0, 1]
query_qubits = [2]

# prepare query register
my_qpe_circ = Circuit().h(query_qubits)

# set unitary
unitary = X

# show small QPE example circuit
my_qpe_circ = my_qpe_circ.qpe(precision_qubits, query_qubits, unitary)
print('QPE CIRCUIT:')
print(my_qpe_circ)
```

    QPE CIRCUIT:
    T  : |0|1|2| 3  |4|     5      |6|

    q0 : -H---U-SWAP---PHASE(-1.57)-H-
              | |      |
    q1 : -H-U-|-SWAP-H-C--------------
            | |
    q2 : -H-U-U-----------------------

    T  : |0|1|2| 3  |4|     5      |6|

As shown in the following code, the two registers can be distributed anywhere across the circuit, with arbitrary indices for the precision and the query registers.

```python
# set qubits
precision_qubits = [1, 3]
query_qubits = [5]

# prepare query register
my_qpe_circ = Circuit().i(range(7))
my_qpe_circ.h(query_qubits)

# set unitary
unitary = X

# show small QPE example circuit
my_qpe_circ = my_qpe_circ.qpe(precision_qubits, query_qubits, unitary)
print('QPE CIRCUIT:')
print(my_qpe_circ)
```

    QPE CIRCUIT:
    T  : |0|1|2|3| 4  |5|     6      |7|

    q0 : -I-----------------------------

    q1 : -I-H---U-SWAP---PHASE(-1.57)-H-
                | |      |
    q2 : -I-----|-|------|--------------
                | |      |
    q3 : -I-H-U-|-SWAP-H-C--------------
              | |
    q4 : -I---|-|-----------------------
              | |
    q5 : -I-H-U-U-----------------------

    q6 : -I-----------------------------

    T  : |0|1|2|3| 4  |5|     6      |7|

As follows, we set up the same circuit, this time implementing the unitary $C-U^{2^k}$, by repeatedly applying the core building block $C-U$.
This operation can be done by setting the parameter ```control_unitary=False``` (default is ```True```).

```python
# set qubits
precision_qubits = [1, 3]
query_qubits = [5]

# prepare query register
my_qpe_circ = Circuit().i(range(7))
my_qpe_circ.h(query_qubits)

# set unitary
unitary = X

# show small QPE example circuit
my_qpe_circ = my_qpe_circ.qpe(precision_qubits, query_qubits, unitary, control_unitary=False)
print('QPE CIRCUIT:')
print(my_qpe_circ)
```

    QPE CIRCUIT:
    T  : |0|1|2|3|4| 5  |6|     7      |8|

    q0 : -I-------------------------------

    q1 : -I-H---U-U-SWAP---PHASE(-1.57)-H-
                | | |      |
    q2 : -I-----|-|-|------|--------------
                | | |      |
    q3 : -I-H-U-|-|-SWAP-H-C--------------
              | | |
    q4 : -I---|-|-|-----------------------
              | | |
    q5 : -I-H-U-U-U-----------------------

    q6 : -I-------------------------------

    T  : |0|1|2|3|4| 5  |6|     7      |8|

In the circuit diagram, we can visually infer the exponents for $k=0,1$, at the expense of a larger circuit depth.

## NUMERICAL TEST EXPERIMENTS

In the following section, we verify that our QFT implementation works as expected with a few test examples:
1. We run QPE with $U=X$ and prepare the eigenstate $|\Psi\rangle = |+\rangle = H|0\rangle$ with phase $\varphi=0$ and eigenvalue $\lambda=1$.
2. We run QPE with $U=X$ and prepare the eigenstate $|\Psi\rangle = |-\rangle = HX|0\rangle$ with phase $\varphi=0.5$ and eigenvalue $\lambda=-1$.
3. We run QPE with $U=X$ and prepare $|\Psi\rangle = |1\rangle = X|0\rangle$ which is *not* an eigenstate of $U$.
Because $|1\rangle = (|+\rangle - |-\rangle)/\sqrt{2}$, we expect to measure both $\varphi=0$ and $\varphi=0.5$ associated with the two eigenstates $|\pm\rangle$.
4. We run QPE with unitary $U=X \otimes Z$, and prepare the query register in the eigenstate $|\Psi\rangle = |+\rangle \otimes |1\rangle = H|0\rangle \otimes Z|0\rangle$.
Here, we expect to measure the phase $\varphi=0.5$ (giving the corresponding eigenvalue $\lambda=-1$).
5. We run QPE with a _random_ two qubit unitary, diagonal in the computational basis, and prepare the query register in the eigenstate $|11\rangle$.
In this case, we should be able to read off the eigenvalue and phase from $U$ and verify QPE gives the right answer (with high probability) up to a small error (that depends on the number of qubits in the precision register).

## HELPER FUNCTIONS FOR NUMERICAL TESTS
Because we will run the same code repeatedly, let's first create a helper function we can use to keep the notebook clean.

```python
def postprocess_qpe_results(out):
    """
    Function to postprocess dictionary returned by run_qpe

    Args:
        out: dictionary containing results/information associated with QPE run as produced by run_qpe
    """

    # unpack results
    circ = out['circuit']
    measurement_counts = out['measurement_counts']
    bitstring_keys = out['bitstring_keys']
    probs_values = out['probs_values']
    precision_results_dic = out['precision_results_dic']
    phases_decimal = out['phases_decimal']
    eigenvalues = out['eigenvalues']

    # print the circuit
    print('Printing circuit:')
    print(circ)

    # print measurement results
    print('Measurement counts:', measurement_counts)

    # plot probabalities
    plt.bar(bitstring_keys, probs_values);
    plt.xlabel('bitstrings');
    plt.ylabel('probability');
    plt.xticks(rotation=90);

    # print results
    print('Results in precision register:', precision_results_dic)
    print('QPE phase estimates:', phases_decimal)
    print('QPE eigenvalue estimates:', np.round(eigenvalues, 5))
```

### NUMERICAL TEST EXAMPLE 1

First, apply the QPE algorithm to the simple single-qubit unitary $U=X$, with eigenstate $|\Psi\rangle = |+\rangle = H|0\rangle$. Here, we expect to measure the phase $\varphi=0$ (giving the corresponding eigenvalue $\lambda=1$).
We show that this result stays the same as we increase the number of qubits $n$ for the top register.

```python
# Set total number of precision qubits: 2
number_precision_qubits = 2

# Define the set of precision qubits
precision_qubits = range(number_precision_qubits)

# Define the query qubits. We'll have them start after the precision qubits
query_qubits = [number_precision_qubits]

# State preparation for eigenstate of U=X
query = Circuit().h(query_qubits)

# Run the test with U=X
out = run_qpe(X, precision_qubits, query_qubits, query, device)

# Postprocess results
postprocess_qpe_results(out)
```

Next, check that we get the same result for a larger precision (top) register.

```python
# Set total number of precision qubits: 3
number_precision_qubits = 3

# Define the set of precision qubits
precision_qubits = range(number_precision_qubits)

# Define the query qubits. We'll have them start after the precision qubits
query_qubits = [number_precision_qubits]

# State preparation for eigenstate of U=X
query = Circuit().h(query_qubits)

# Run the test with U=X
out = run_qpe(X, precision_qubits, query_qubits, query, device)

# Postprocess results
postprocess_qpe_results(out)
```

### NUMERICAL TEST EXAMPLE 2

Next, apply the QPE algorithm to the simple single-qubit unitary $U=X$, with eigenstate $|\Psi\rangle = |-\rangle = HX|0\rangle$.
Here, we expect to measure the phase $\varphi=0.5$ (giving the corresponding eigenvalue $\lambda=-1$).

```python
# Set total number of precision qubits: 2
number_precision_qubits = 2

# Define the set of precision qubits
precision_qubits = range(number_precision_qubits)

# Define the query qubits. We'll have them start after the precision qubits
query_qubits = [number_precision_qubits]

# State preparation for eigenstate of U=X
query = Circuit().x(query_qubits).h(query_qubits)

# Run the test with U=X
out = run_qpe(X, precision_qubits, query_qubits, query, device)

# Postprocess results
postprocess_qpe_results(out)
```

### NUMERICAL TEST EXAMPLE 3

Next, apply the QPE algorithm again to the simple single-qubit unitary $U=X$, but we initialize the query register in the state $|\Psi\rangle = |1\rangle$ which is *not* an eigenstate of $U$.
Here, following the previous discussion, we expect to measure the phases $\varphi=0, 0.5$ (giving the corresponding eigenvalue $\lambda=\pm 1$). Accordingly, here we set ```items_to_keep=2```.

```python
# Set total number of precision qubits: 2
number_precision_qubits = 2

# Define the set of precision qubits
precision_qubits = range(number_precision_qubits)

# Define the query qubits. We'll have them start after the precision qubits
query_qubits = [number_precision_qubits]

# State preparation for |1>, which is not an eigenstate of U=X
query = Circuit().x(query_qubits)

# Run the test with U=X
out = run_qpe(X, precision_qubits, query_qubits, query, device, items_to_keep=2)

# Postprocess results
postprocess_qpe_results(out)
```

### NUMERICAL TEST EXAMPLE 4

Next, apply the QPE algorithm to the two-qubit unitary $U=X \otimes Z$, and prepare the query register in the eigenstate $|\Psi\rangle = |+\rangle \otimes |1\rangle = H|0\rangle \otimes X|0\rangle$.
Here, we expect to measure the phase $\varphi=0.5$ (giving the corresponding eigenvalue $\lambda=-1$).

```python
# set unitary matrix U
u1 = np.kron(X, Id)
u2 = np.kron(Id, Z)
unitary = np.dot(u1, u2)
print('Two-qubit unitary (XZ):\n', unitary)

# get example eigensystem
eig_values, eig_vectors = np.linalg.eig(unitary)
print('Eigenvalues:', eig_values)
# print('Eigenvectors:', eig_vectors)
```

    Two-qubit unitary (XZ):
     [[ 0.  0.  1.  0.]
     [ 0.  0.  0. -1.]
     [ 1.  0.  0.  0.]
     [ 0. -1.  0.  0.]]
    Eigenvalues: [ 1. -1.  1. -1.]

```python
# Set total number of precision qubits: 2
number_precision_qubits = 2

# Define the set of precision qubits
precision_qubits = range(number_precision_qubits)

# Define the query qubits. We'll have them start after the precision qubits
query_qubits = [number_precision_qubits, number_precision_qubits+1]

# State preparation for eigenstate |+,1> of U=X \otimes Z
query = Circuit().h(query_qubits[0]).x(query_qubits[1])

# Run the test with U=X
out = run_qpe(unitary, precision_qubits, query_qubits, query, device)

# Postprocess results
postprocess_qpe_results(out)
```

### NUMERICAL TEST EXAMPLE 5

In this example, we choose the unitary to be a _random_ two-qubit unitary, diagonal in the computational basis. We initialize the query register to be in the eigenstate $|11\rangle$ of $U$, which we can prepare using that $|11\rangle = X\otimes X|00\rangle$.
In this case we should be able to read off the eigenvalue and phase from $U$ and verify that QPE gives the right answer.

```python
# Generate a random 2 qubit unitary matrix:
from scipy.stats import unitary_group

# Fix random seed for reproducibility
np.random.seed(seed=42)

# Get random two-qubit unitary
random_unitary = unitary_group.rvs(2**2)

# Let's diagonalize this
evals = np.linalg.eig(random_unitary)[0]

# Since we want to be able to read off the eigenvalues of the unitary in question
# let's choose our unitary to be diagonal in this basis
unitary = np.diag(evals)

# Check that this is indeed unitary, and print it out:
print('Two-qubit random unitary:\n', np.round(unitary, 3))
print('Check for unitarity: ', np.allclose(np.eye(len(unitary)), unitary.dot(unitary.T.conj())))

# Print eigenvalues
print('Eigenvalues:', np.round(evals, 3))
```

    Two-qubit random unitary:
     [[-0.078+0.997j  0.   +0.j     0.   +0.j     0.   +0.j   ]
     [ 0.   +0.j    -0.987-0.159j  0.   +0.j     0.   +0.j   ]
     [ 0.   +0.j     0.   +0.j     0.192-0.981j  0.   +0.j   ]
     [ 0.   +0.j     0.   +0.j     0.   +0.j     0.747-0.665j]]
    Check for unitarity:  True
    Eigenvalues: [-0.078+0.997j -0.987-0.159j  0.192-0.981j  0.747-0.665j]

When we execute the QPE circuit, we expect the following (approximate) result for the eigenvalue estimate:

```python
print('Target eigenvalue:', np.round(evals[-1], 3))
```

    Target eigenvalue: (0.747-0.665j)

```python
# Set total number of precision qubits
number_precision_qubits = 3

# Define the set of precision qubits
precision_qubits = range(number_precision_qubits)

# Define the query qubits. We'll have them start after the precision qubits
query_qubits = [number_precision_qubits, number_precision_qubits+1]

# State preparation for eigenstate |1,1> of diagonal U
query = Circuit().x(query_qubits[0]).x(query_qubits[1])

# Run the test with U=X
out = run_qpe(unitary, precision_qubits, query_qubits, query, device)

# Postprocess results
postprocess_qpe_results(out)

# compare output to exact target values
print('Target eigenvalue:', np.round(evals[-1], 3))
```

We can easily improve the precision of our parameter estimate by increasing the number of qubits in the precision register, as shown in the following example.

```python
# Set total number of precision qubits
number_precision_qubits = 10

# Define the set of precision qubits
precision_qubits = range(number_precision_qubits)

# Define the query qubits. We'll have them start after the precision qubits
query_qubits = [number_precision_qubits, number_precision_qubits+1]

# State preparation for eigenstate |1,1> of diagonal U
query = Circuit().x(query_qubits[0]).x(query_qubits[1])

# Run the test with U=X
out = run_qpe(unitary, precision_qubits, query_qubits, query, device)

# Postprocess results
eigenvalues = out['eigenvalues']
print('QPE eigenvalue estimates:', np.round(eigenvalues, 5))

# compare output to exact target values
print('Target eigenvalue:', np.round(evals[-1], 5))
```

    QPE eigenvalue estimates: [0.74506-0.667j]
    Target eigenvalue: (0.74699-0.66484j)

---
## APPENDIX

```python
# Check SDK version
# alternative: braket.__version__
!pip show amazon-braket-sdk | grep Version
```

    Version: 0.6.0

## Details of the ```utiles_qpe.py``` module

### Imports, including inverse QFT

```python
# general imports
import numpy as np
import math
from collections import Counter
from datetime import datetime
import pickle

# AWS imports: Import Braket SDK modules
from braket.circuits import Circuit, circuit

# local imports
from utils_qft import inverse_qft
```

### QPE Subroutine

```python
@circuit.subroutine(register=True)
def controlled_unitary(control, target_qubits, unitary):
    """
    Construct a circuit object corresponding to the controlled unitary

    Args:
        control: The qubit on which to control the gate

        target_qubits: List of qubits on which the unitary U acts

        unitary: matrix representation of the unitary we wish to implement in a controlled way
    """

    # Define projectors onto the computational basis
    p0 = np.array([[1., 0.],
                   [0., 0.]])

    p1 = np.array([[0., 0.],
                   [0., 1.]])

    # Instantiate circuit object
    circ = Circuit()

    # Construct numpy matrix
    id_matrix = np.eye(len(unitary))
    controlled_matrix = np.kron(p0, id_matrix) + np.kron(p1, unitary)

    # Set all target qubits
    targets = [control] + target_qubits

    # Add controlled unitary
    circ.unitary(matrix=controlled_matrix, targets=targets)

    return circ

@circuit.subroutine(register=True)
def qpe(precision_qubits, query_qubits, unitary, control_unitary=True):
    """
    Function to implement the QPE algorithm using two registers for precision (read-out) and query.
    Register qubits need not be contiguous.

    Args:
        precision_qubits: list of qubits defining the precision register

        query_qubits: list of qubits defining the query register

        unitary: Matrix representation of the unitary whose eigenvalues we wish to estimate

        control_unitary: Optional boolean flag for controlled unitaries,
                         with C-(U^{2^k}) by default (default is True),
                         or C-U controlled-unitary (2**power) times
    """
    qpe_circ = Circuit()

    # Get number of qubits
    num_precision_qubits = len(precision_qubits)
    num_query_qubits = len(query_qubits)

    # Apply Hadamard across precision register
    qpe_circ.h(precision_qubits)

    # Apply controlled unitaries. Start with the last precision_qubit, and end with the first
    for ii, qubit in enumerate(reversed(precision_qubits)):
        # Set power exponent for unitary
        power = ii

        # Alterantive 1: Implement C-(U^{2^k})
        if control_unitary:
            # Define the matrix U^{2^k}
            Uexp = np.linalg.matrix_power(unitary,2**power)

            # Apply the controlled unitary C-(U^{2^k})
            qpe_circ.controlled_unitary(qubit, query_qubits, Uexp)
        # Alterantive 2: One can instead apply controlled-unitary (2**power) times to get C-U^{2^power}
        else:
            for _ in range(2**power):
                qpe_circ.controlled_unitary(qubit, query_qubits, unitary)

    # Apply inverse qft to the precision_qubits
    qpe_circ.inverse_qft(precision_qubits)

    return qpe_circ
```

### QPE postprocessing helper functions

```python
# helper function to remove query bits from bitstrings
def substring(key, precision_qubits):
    """
    Helper function to get substring from keys for dedicated string positions as given by precision_qubits.
    This function is necessary to allow for arbitrary qubit mappings in the precision and query registers
    (that is, so that the register qubits need not be contiguous.)

    Args:
        key: string from which we want to extract the substring supported only on the precision qubits

        precision_qubits: List of qubits corresponding to precision_qubits.
                          Currently assumed to be a list of integers corresponding to the indices of the qubits
    """
    short_key = ''
    for idx in precision_qubits:
        short_key = short_key + key[idx]

    return short_key

# helper function to convert binary fractional to decimal
# reference: https://www.geeksforgeeks.org/convert-binary-fraction-decimal/
def binaryToDecimal(binary):
    """
    Helper function to convert binary string (example: '01001') to decimal

    Args:
        binary: string which to convert to decimal fraction
    """

    length = len(binary)
    fracDecimal = 0

    # Convert fractional part of binary to decimal equivalent
    twos = 2

    for ii in range(length):
        fracDecimal += ((ord(binary[ii]) - ord('0')) / twos);
        twos *= 2.0

    # return fractional part
    return fracDecimal

# helper function for postprocessing based on measurement shots
def get_qpe_phases(measurement_counts, precision_qubits, items_to_keep=1):
    """
    Get QPE phase estimate from measurement_counts for given number of precision qubits

    Args:
        measurement_counts: measurement results from a device run

        precision_qubits: List of qubits corresponding to precision_qubits.
                          Currently assumed to be a list of integers corresponding to the indices of the qubits

        items_to_keep: number of items to return (topmost measurement counts for precision register)
    """

    # Aggregate the results (that is, ignore the query register qubits):

    # First get bitstrings with corresponding counts for precision qubits only
    bitstrings_precision_register = [substring(key, precision_qubits)  for key in measurement_counts.keys()]
    # Then keep only the unique strings
    bitstrings_precision_register_set = set(bitstrings_precision_register)
    # Cast as a list for later use
    bitstrings_precision_register_list = list(bitstrings_precision_register_set)

    # Now create a new dict to collect measurement results on the precision_qubits.
    # Keys are given by the measurement count substrings on the register qubits. Initialize the counts to zero.
    precision_results_dic = {key: 0 for key in bitstrings_precision_register_list}

    # Loop over all measurement outcomes
    for key in measurement_counts.keys():
        # Save the measurement count for this outcome
        counts = measurement_counts[key]
        # Generate the corresponding shortened key (supported only on the precision_qubits register)
        count_key = substring(key, precision_qubits)
        # Add these measurement counts to the corresponding key in our new dict
        precision_results_dic[count_key] += counts

    # Get topmost values only
    c = Counter(precision_results_dic)
    topmost= c.most_common(items_to_keep)
    # get decimal phases from bitstrings for topmost bitstrings
    phases_decimal = [binaryToDecimal(item[0]) for item in topmost]

    # Get decimal phases from bitstrings for all bitstrings
    # number_precision_qubits = len(precision_qubits)
    # Generate binary decimal expansion
    # phases_decimal = [int(key, 2)/(2**number_precision_qubits) for key in precision_results_dic]
    # phases_decimal = [binaryToDecimal(key) for key in precision_results_dic]

    return phases_decimal, precision_results_dic
```

### Run QPE experiments:

```python
def run_qpe(unitary, precision_qubits, query_qubits, query_circuit,
            device, items_to_keep=1, shots=1000, save_to_pck=False):
    """
    Function to run QPE algorithm end-to-end and return measurement counts.

    Args:
        precision_qubits: list of qubits defining the precision register

        query_qubits: list of qubits defining the query register

        unitary: Matrix representation of the unitary whose eigenvalues we wish to estimate

        query_circuit: query circuit for state preparation of query register

        items_to_keep: (optional) number of items to return (topmost measurement counts for precision register)

        device: Braket device backend

        shots: (optional) number of measurement shots (default is 1000)

        save_to_pck: (optional) save results to pickle file if True (default is False)
    """

    # get size of precision register and total number of qubits
    number_precision_qubits = len(precision_qubits)
    num_qubits = len(precision_qubits) + len(query_qubits)

    # Define the circuit. Start by copying the query_circuit, then add the QPE:
    circ = query_circuit
    circ.qpe(precision_qubits, query_qubits, unitary)

    # Add desired results_types
    circ.probability()

    # Run the circuit with all zeros input.
    # The query_circuit subcircuit generates the desired input from all zeros.
    # The following code executes the correct device.run call, depending on whether the backend is local or managed (cloud-based)
    if device.name == 'DefaultSimulator':
        task = device.run(circ, shots=shots)
    else:
        task = device.run(circ, shots=shots)

    # get result for this task
    result = task.result()

    # get metadata
    metadata = result.task_metadata

    # get output probabilities (see result_types above)
    probs_values = result.values[0]

    # get measurement results
    measurements = result.measurements
    measured_qubits = result.measured_qubits
    measurement_counts = result.measurement_counts
    measurement_probabilities = result.measurement_probabilities

    # bitstrings
    format_bitstring = '{0:0' + str(num_qubits) + 'b}'
    bitstring_keys = [format_bitstring.format(ii) for ii in range(2**num_qubits)]

    # QPE postprocessing
    phases_decimal, precision_results_dic = get_qpe_phases(measurement_counts, precision_qubits, items_to_keep)
    eigenvalues = [np.exp(2*np.pi*1j*phase) for phase in phases_decimal]

    # aggregate results
    out = {'circuit': circ,
           'task_metadata': metadata,
           'measurements': measurements,
           'measured_qubits': measured_qubits,
           'measurement_counts': measurement_counts,
           'measurement_probabilities': measurement_probabilities,
           'probs_values': probs_values,
           'bitstring_keys': bitstring_keys,
           'precision_results_dic': precision_results_dic,
           'phases_decimal': phases_decimal,
           'eigenvalues': eigenvalues}

    if save_to_pck:
        # store results: dump output to pickle with timestamp in filename
        time_now = datetime.strftime(datetime.now(), '%Y%m%d%H%M%S')
        results_file = 'results-'+time_now+'.pck'
        pickle.dump(out, open(results_file, "wb"))
        # you can load results as follows
        # out = pickle.load(open(results_file, "rb"))

    return out
```

---
## REFERENCES

[1] Wikipedia: https://en.wikipedia.org/wiki/Quantum_phase_estimation_algorithm

[2] Nielsen, Michael A., Chuang, Isaac L. (2010). Quantum Computation and Quantum Information (2nd ed.). Cambridge: Cambridge University Press.


# Stagnation flow

This notebook will use Dedalus to create a minimal working example of the solution to viscous flow past a stagnation point, and the volume penalized approximation

\begin{align}
(\partial_{x} u)^2 - \partial_x^2 u \, u - \frac{1}{\text{Re}} \partial_x^3 u -  \frac{1}{\text{Re}\,\varepsilon^2}\Gamma u &= 1
\end{align}

where the penalty mask $\Gamma$ can be chosen by shifting and rescaling optimal masks $\Gamma^*$

\begin{align}
\Gamma(x) &= \Gamma^*\left(\frac{x - \varepsilon \ell}{\varepsilon \delta}\right)
\end{align}

and the physical solution $u$ satisfies the boundary conditions

\begin{align}
x &= -1 & u &= 0\\
x &= -1 & \partial_ x u &= 0\\
x &\to +\infty & \partial_x u &\to 1
\end{align}

# Imports

```python
import numpy as np
import matplotlib.pyplot as plt
import dedalus.public as de

from matplotlib import rc
rc('font',**{'family':'serif','serif':['Computer Modern Roman']})
rc('text', usetex=True)
```

# Ideal no-slip solution

We will first solve the problem numerically using true no-slip boundaries

```python
N = 256
Re = 1

xbasis = de.Chebyshev('x', N, interval=(0,10), dealias=3/2)
domain = de.Domain([xbasis], grid_dtype=np.float64)
x0 = xbasis.grid(*domain.dealias)

stag = de.NLBVP(domain,variables=['u','ux','uxx'])
stag.meta[:]['x']['dirichlet'] = True
stag.parameters['Re'] = Re
stag.add_equation('dx(u) - ux = 0')
stag.add_equation('dx(ux) - uxx = 0')
stag.add_equation('dx(uxx)/Re = -1 - uxx*u + ux*ux')
stag.add_bc('right(ux) = 1')
stag.add_bc('left(u) = 0')
stag.add_bc('left(ux) = 0')

solver = stag.build_solver()
u0, ux0, uxx0 = (solver.state[name] for name in stag.variables)
for field in [u0, ux0, uxx0]: field.set_scales(domain.dealias)

u0['g'], ux0['g'], uxx0['g'] = x0, 1, 0

tolerance = 1e-10
pert = solver.perturbations.data
pert.fill(1+tolerance)

while np.sum(np.abs(pert)) > tolerance:
    solver.newton_iteration()
```

# Volume penalized solution

We now solve the volume penalized problem.

```python
N = [32,64]
Re = 1
η = 0.01
ϵ = np.sqrt(η/Re)
l = ϵ

xb0 = de.Chebyshev('x0', N[0], interval=(-1,l), dealias=3/2)
xb1 = de.Chebyshev('x1', N[1], interval=(l,5), dealias=3/2)
xbasis = de.Compound('x',[xb0,xb1])
domain = de.Domain([xbasis], grid_dtype=np.float64)
x = xbasis.grid(*domain.dealias)

Γ = domain.new_field(scales=domain.dealias)
Γ['g'] = 1.*(x<l)

stag = de.NLBVP(domain,variables=['u','ux','uxx'])
stag.meta[:]['x']['dirichlet'] = True
stag.parameters['Re'] = Re
stag.parameters['Γ'] = Γ
stag.parameters['ε'] = ϵ
stag.add_equation('dx(u) - ux = 0')
stag.add_equation('dx(ux) - uxx = 0')
stag.add_equation('dx(uxx)/Re - (Γ/(Re*ε**2))*ux = - 1 - uxx*u + ux*ux')
stag.add_bc('right(ux) = 1')
stag.add_bc('left(u) = 0')
stag.add_bc('left(ux) = 0')

solver = stag.build_solver()
u, ux, uxx = (solver.state[name] for name in stag.variables)
for field in [u, ux, uxx]: field.set_scales(domain.dealias)

u['g'], ux['g'], uxx['g'] = x, 1, 0

tolerance = 1e-10
pert = solver.perturbations.data
pert.fill(1+tolerance)

while np.sum(np.abs(pert)) > tolerance:
    solver.newton_iteration()
```

```python
# Comparison of penalized and reference solutions
fig, ax = plt.subplots()
ax.plot(x,u['g'],'C2',label='Penalized')
ax.plot(x0,u0['g'],'k--',label='Reference')
ax.fill_between(x,0,1.5*Γ['g'],color='lightgray')
ax.set(aspect=1,xlim=[-1,2],ylim=[0,1.5],yticks=np.arange(5),
       xlabel='$x$',ylabel='$u$')
ax.spines['right'].set_visible(False)
ax.spines['top'].set_visible(False)
ax.legend()
```


# Intro: Linear/Nonlinear Optimization in Python

Python 提供多种库来解决最优化的问题，对线性规划，python提供了linprog,也可以直接使用minimize. 如以下的问题：
$$ \large 目标函数：\begin{equation}\begin{array}{l} min  f=-\ x_1-\ x_2-\ x_3 \end{array}\end{equation}  $$
$$ \large s.t. \left\{
\begin{array}{rcl}
7 \ x_1 +3 \ x_2+9 \ x_3 \leq 1\\
8 \ x_1 +5 \ x_2+4 \ x_3 \leq 1\\
6 \ x_1 +9 \ x_2+5 \ x_3 \leq 1\\
x_1,x_2,x_3 \geq 0
\end{array} \right. $$

```python
import numpy as np
from scipy import optimize
import pandas as pd
```

```python
f = np.array([-1, -1, -1])
A_ub = np.array([[7, 3, 9],[8, 5, 4],[6, 9, 5]])
b_ub = np.array([1, 1, 1])
lb =((0,100),(0,100),(0,100));
optimize.linprog(f, A_ub, b_ub,bounds=lb)
```

         con: array([], dtype=float64)
         fun: -0.15415019419978693
     message: 'Optimization terminated successfully.'
         nit: 9
       slack: array([9.32575905e-09, 2.64429326e-07, 9.03222153e-09])
      status: 0
     success: True
           x: array([0.08695645, 0.03557314, 0.0316206 ])

Alternatively

```python
v = np.array([-1, -1, -1])
def f(x,v):
    return np.sum(x*v)
initial=np.array([0,0,0])
cons1={'type':'eq','fun':lambda x:f(x,A_ub[0])-b_ub[0]}
cons2={'type':'eq','fun':lambda x:f(x,A_ub[1])-b_ub[1]}
cons3={'type':'eq','fun':lambda x:f(x,A_ub[2])-b_ub[2]}
res=optimize.minimize(f,initial,v,constraints=[cons1,cons2,cons3],bounds=((0,100),(0,100),(0,100)))
res
```

         fun: -0.15415019762845847
         jac: array([-1., -1., -1.])
     message: 'Optimization terminated successfully.'
        nfev: 10
         nit: 2
        njev: 2
      status: 0
     success: True
           x: array([0.08695652, 0.03557312, 0.03162055])

```python
fun = lambda x: -1*x[0] - 1*x[1] - 1*x[2]
cons = ({'type': 'eq', 'fun': lambda x:  7*x[0] + 3*x[1] + 9*x[2] - 1},
        {'type': 'eq', 'fun': lambda x: 8*x[0] + 5*x[1] + 4*x[2]- 1},
        {'type': 'eq', 'fun': lambda x: 6*x[0] + 9*x[1] + 5*x[2] - 1})
bnds = ((0, 100), (0, 100), (0,1))
res = optimize.minimize(fun, (0,0,0), bounds=bnds, constraints=cons)
res
```

         fun: -0.15415019762845847
         jac: array([-1., -1., -1.])
     message: 'Optimization terminated successfully.'
        nfev: 10
         nit: 2
        njev: 2
      status: 0
     success: True
           x: array([0.08695652, 0.03557312, 0.03162055])

又或者可以用pulp库：使用pulp库之前，需要在cmd中输入如下的安装语句：
pip install pulp

```python
from pulp import *
```

```python
my_LpProblem = LpProblem("test1",LpMinimize)
x = LpVariable("x",0)
y = LpVariable("y",0)
z = LpVariable("z",0)

my_LpProblem += -x-y-z, "obj"
my_LpProblem += 7*x + 3*y + 9*z <= 1, "c1"
my_LpProblem +=  8*x + 5*y + 4*z <= 1, "c2"
my_LpProblem += 6*x + 9*y +5*z <= 1, "c3"

my_LpProblem.solve()

print("Status:",LpStatus[my_LpProblem.status])

for v in my_LpProblem.variables():
    print(v.name,"=", v.varValue)
```

    Status: Optimal
    x = 0.086956522
    y = 0.035573123
    z = 0.031620553

总结：optimize库提供的linprog可以用于简单的线性规划问题，pulp库可以应用于较为复杂的线性规划问题，
pulp库的使用更加直观。optimize库也提供了minimize函数，既可以用于线性规划，也可以用于非线性规划，
以下面的例子来说明minimize在非线性最优化的应用。

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```


# Almgren and Chriss Model For Optimal Execution of Portfolio Transactions

### Introduction

We consider the execution of portfolio transactions with the aim of minimizing a combination of risk and transaction costs arising from permanent and temporary market impact. As an example, assume that you have a certain number of stocks that you want to sell within a given time frame. If you place this sell order directly to the market as it is, transaction costs may rise due to temporary market impact. On the other hand, if you split up into pieces in time, cost may rise due to volatility in the stock price.

[Almgren and Chriss](https://cims.nyu.edu/~almgren/papers/optliq.pdf) provided a solution to this problem by assuming the permanent and temporary market impact functions are linear functions of the rate of trading, and that stock prices follow a discrete arithmetic random walk.

In this notebook, we will take a look at the model used by Almgren and Chriss to solve the optimal liquidation problem. We will start by stating the formal definitions of *trading trajectory*, *trading list*, and *trading strategy* for liquidating a single stock.

### Trading Trajectory, Trading List, and Trading Strategy

We define trading trajectory, trading list, and trading strategy just as Almgren and Chriss did in their [paper](https://cims.nyu.edu/~almgren/papers/optliq.pdf). Suppose we hold $X$ shares of a stock that we want to liquidate before time $T$. Divide $T$ into $N$ intervals of length $\tau=\frac{T}{N}$ and define:

- $t_k = k\tau$ to be discrete times, where $k = 0,..,N$.

- A **trading trajectory** to be the list $(x_0,..,x_N)$, where $x_k$ is the number of shares we plan to hold at time $t_k$. We require that our initial position $x_0 = X$, and that at liquidation time $T$, $x_N = 0$.

- A **trading list** to be $(n_1,..,n_N)$, $n_k =  x_{k-1} - x_k$ as the number of shares that we will sell between times $t_{k-1}$ and $t_k$.

- A **trading strategy** as a rule for determining $n_k$ from the information available at time $t_{k-1}$.

Below, we can see a visual example of a trading trajectory, for $N = 12$.

## Price Dynamics

We will assume that the stock price evolves according to a discrete arithmetic random walk:

\begin{equation}
S_k = S_{k-1} + \sigma \tau^{1/2} \xi_k
\end{equation}

for $k = 1,..,N$ and where:

\begin{equation}
S_k = \text{ stock price at time $k$}\hspace{21.6cm}\\
\sigma = \text{ standard deviation of the fluctuations in stock price}\hspace{16.3cm}\\
\tau = \text{ length of discrete time interval}\hspace{20.2cm}\\
\xi_k = \text{ draws from independent random variables}\hspace{17.8cm}
\end{equation}

We will denote the initial stock price as $S_0$. The role of $\xi_k$ is to simulate random price fluctuations using random numbers drawn from a Normal Gaussian distribution with zero mean and unit variance. The code below shows us what this price model looks like, for an initial stock price of $S_0 =$ \$50 dollars, a standard deviation of price fluctuations of $\sigma = 0.379$, and a discrete time interval of $\tau = 1$.

```python
%matplotlib inline

import matplotlib.pyplot as plt

# Add-on : Hide Matplotlib deprecate warnings
import warnings
warnings.filterwarnings("ignore")
# High resolution plot outputs for retina display
%config InlineBackend.figure_format = 'retina'

import utils

# Set the default figure size
plt.rcParams['figure.figsize'] = [17.0, 7.0]

# Set the number of days to follow the stock price
n_days = 100

# Plot the stock price as a function of time
utils.plot_price_model(seed = 0, num_days = n_days)
```

## Market Impact

As we learned previously the price of a stock is affected by market impact that occurs every time we sell a stock. In their model, Almgren and Chriss distinguish between two types of market impact, permanent and temporary market impact. We will now add these two factors into our price model.

### Permanent Impact

Permanent market impact refers to changes in the equilibrium price of a stock as a direct function of our trading. Permanent market impact is called *permanent* because its effect persists for the entire liquidation period, $T$. We will denote the permanent price impact as $g(v)$, and will add it to our price model:

\begin{equation}
S_k = S_{k-1} + \sigma \tau^{1/2} \xi_k - \tau g\left(\frac{n_k}{\tau}\right)
\end{equation}

Here, we assumed the permanent impact function, $g(v)$, is a linear function of the trading rate, $v = n_k / \tau$. We will take $g(v)$ to have the form:

\begin{equation}
g(v) = \gamma \left(\frac{n_k}{\tau}\right)
\end{equation}

where $\gamma$ is a constant and has units of (\$/share${}^2$). Replacing this in the above equation we get:

\begin{equation}
S_k = S_{k-1} + \sigma \tau^{1/2} \xi_k - \gamma n_k
\end{equation}

With this form, we can see that for each $n$ shares that we sell, we will depress the stock price permanently by $n\gamma$, regardless of the time we take to sell the stocks.

### Temporary Impact

Temporary market impact refers to temporary imbalances in supply and demand caused by our trading. This leads to temporary price movements away from equilibrium. Temporary market impact is called *temporary* because its effect
dissipates by the next trading period. We will denote the temporary price impact as $h(v)$. Given this, the actual stock price at time $k$ is given by:

\begin{equation}
\tilde{S_k} = S_{k-1} - h\left(\frac{n_k}{\tau}\right)
\end{equation}

Where, we have again assumed the temporary impact function, $h(v)$, is a linear function of the trading rate, $v = n_k / \tau$. We will take $h(v)$ to have the form:

\begin{equation}
h(v) = \epsilon \mbox{ sign}(n_k) + \eta \left(\frac{n_k}{\tau}\right)
\end{equation}

where $\epsilon$ and $\eta$ are constants with units (\$/share) and (\$ time/share${}^2$), respectively. It is important to note that $h(v)$ does not affect the price $S_k$.

## Capture

We define the **Capture** to be the total profits resulting from trading along a particular trading trajectory, upon completion of all trades. We can compute the capture via:

\begin{equation}
\sum\limits_{k=1}^{N} n_k \tilde{S_k} = X S_0 + \sum\limits_{k=1}^{N} \left(\sigma \tau^{1/2} \xi_k - \tau g\left(\frac{n_k}{\tau}\right)\right) x_k - \sum\limits_{k=1}^{N} n_k h\left(\frac{n_k}{\tau}\right)
\end{equation}

As we can see this is the sum of the product of the number of shares $n_k$ that we sell in each time interval, times the effective price per share $\tilde{S_k}$ received on that sale.

## Implementation Shortfall

We define the **Implementation Shortfall** as the total cost of trading and is given by:

\begin{equation}
I_s = X S_0 - \sum_{k = 1}^N n_k \tilde{S_k}
\end{equation}

This is what we seek to minimize when determining the best trading strategy!

Note that since $\xi_k$ is random, so is the implementation shortfall. Therefore, we have to frame the minimization problem in terms of the expectation value of the shortfall and its corresponding variance. We'll refer to $E(x)$ as the expected shortfall and $V(x)$ as the variance of the shortfall. Simplifying the above equation for $I_s$, is easy to see that:

\begin{equation}
E(x) = \sum\limits_{k=1}^{N} \tau x_k g\left(\frac{n_k}{\tau}\right) + \sum\limits_{k=1}^{N} n_k h\left(\frac{n_k}{\tau}\right)
\end{equation}

and

\begin{equation}
V(x) = \sigma^2 \sum\limits_{k=1}^{N} \tau {x_k}^2
\end{equation}

The units of $E(x)$ are dollars and the units of $V(x)$ are dollars squared. So now, we can reframe our minimization problem in terms of $E(x)$ and $V(x)$.
For a given level of variance of shortfall, $V(x)$, we seek to minimize the expectation of shortfall, $E(x)$. In the next section we will see how to solve this problem.

## Utility Function

Our goal now is to find the strategy that has the minimum expected shortfall $E(x)$ for a given maximum level of variance $V(x) \ge 0$. This constrained optimization problem can be solved by introducing a Lagrange multiplier $\lambda$. Therefore, our problem reduces to finding the trading strategy that minimizes the **Utility Function** $U(x)$:

\begin{equation}
U(x) = E(x) + \lambda V(x)
\end{equation}

The parameter $\lambda$ is referred to as **trader’s risk aversion** and controls how much we penalize the variance relative to the expected shortfall.

The intuition of this utility function can be thought of as follows. Consider a stock which exhibits high price volatility and thus a high risk of price movement away from the equilibrium price. A risk averse trader would prefer to trade a large portion of the volume immediately, causing a known price impact, rather than risk trading in small increments at successively adverse prices. Alternatively, if the price is expected to be stable over the liquidation period, the trader would rather split the trade into smaller sizes to avoid price impact. This trade-off between speed of execution and risk of price movement is ultimately what governs the structure of the resulting trade list.

# Optimal Trading Strategy

Almgren and Chriss solved the above problem and showed that for each value
of risk aversion there is a uniquely determined optimal execution strategy. The details of their derivation is discussed in their [paper](https://cims.nyu.edu/~almgren/papers/optliq.pdf). Here, we will just state the general solution.

The optimal trajectory is given by:

\begin{equation}
x_j = \frac{\sinh \left( \kappa \left( T-t_j\right)\right)}{ \sinh (\kappa T)}X, \hspace{1cm}\text{ for } j=0,...,N
\end{equation}

and the associated trading list:

\begin{equation}
n_j = \frac{2 \sinh \left(\frac{1}{2} \kappa \tau \right)}{ \sinh \left(\kappa T\right) } \cosh \left(\kappa \left(T - t_{j-\frac{1}{2}}\right)\right) X, \hspace{1cm}\text{ for } j=1,...,N
\end{equation}

where $t_{j-1/2} = (j-\frac{1}{2}) \tau$. The expected shortfall and variance of the optimal trading strategy are given by:

In the above equations $\kappa$ is given by:

\begin{align*}
&\kappa = \frac{1}{\tau}\cosh^{-1}\left(\frac{\tau^2}{2}\tilde{\kappa}^2 + 1\right)
\end{align*}

where:

\begin{align*}
&\tilde{\kappa}^2 = \frac{\lambda \sigma^2}{\tilde{\eta}} = \frac{\lambda \sigma^2}{\eta \left(1-\frac{\gamma \tau}{2 \eta}\right)}
\end{align*}

```python

```


```python
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from sklearn.linear_model import LinearRegression
```

# Gradient Descent

## A Blindfolded Hiker

Imagine a person who is blindfolded who is sitting at the bottom of a hill (or anywhere on the hill really). Their goal is to get to the top of the hill. How would they do it?

Well, depending on wherever they're currently standing, they would detect the slope of the hill that they're standing on and take a step in an upwards direction. With each step our hiker feels their way forward and re-evaluates the slope of the hill where they are standing before taking anther step. Through this iterative process of analyzing the slope of the hill and taking steps upwards the hiker makes progress up the hill.

Our climber continues this process until they feel like they're on flat ground. If they accidentally overshoot the summit they would feel it due to the slope then pointing downwards and would know to turn around and move back up to the highest (flattest) point.

# Gradient Descent is a Blindfolded Hiker

Gradient Descent is an algorithm that is used for finding the optimal values of a function (maximum or minimum). Although we described our hiker man as if he was trying to find a maximum value, Gradient Descent is typically used to find minimum values of a function. Lets learn a little more about how Gradient Descent uses an iterative mathematic approach (like our hiker) to find a function's minimum.

# "Hypothesis" functions.

In Linear Regression we end up estimating parameters $\beta_0, \beta_1, \ldots , \beta_n$. and once we have estimated these parameters we then take them and plug them into our model. This version of our model with real coefficient values is a Hypothesis Function. However, it is not a requirement of hypothesis functions that they use the estimated parameters. They could contain any real values for the betas and still be a hypothesis function. Lets look at some examples. We'll go back to our ice cream sales example for some easy data to demonstrate with.

```python
# Read Data
df = pd.read_csv('https://raw.githubusercontent.com/ryanleeallred/datasets/master/Ice_Cream_Sales.csv')

# Creat Scatterplot
plt.scatter(df.Farenheit, df.Dollars)

X = df.Farenheit[:, np.newaxis]
y = df.Dollars

# Create Hypothesis Function Lines
h1 = [30*x - 500 for x in X]
h2 = [10*x + 0 for x in X]
h3 = [-10*x + 1000 for x in X]
plt.plot(X, h1)
plt.plot(X, h2)
plt.plot(X, h3)

# Graph axes and title
plt.ylabel("Dollars")
plt.xlabel("Farenheit")
plt.title("Hypothesis Functions")

# Plot Hypothesis Functions
plt.show()
```

The above are all Hypothesis functions, they don't necessarily fit our data the best, but they are created using the same model, they just have different (arbitrary) parameters for $\beta_0$ and $\beta_1$. We'll represent hypothesis functions generally as $h(x)$.

The above hypotheis functions all follow the form:

\begin{align}
  h(x) = \theta_0 + \theta_1X
\end{align}

Basically we're assuming that the function that best fits the data is linear (a line) and hypothesizing the $\theta$ values that define that line.

## Given the hypothesis functions below, which is closest to the optimum?

How do you know? How do you measure the quality of a hypothesis?

```python
# Creat Scatterplot
plt.scatter(df.Farenheit, df.Dollars)

X = df.Farenheit[:, np.newaxis]
y = df.Dollars

# Create Hypothesis Function Lines
h1 = [10*x - 500 for x in X]
h2 = [12*x - 510 for x in X]
h3 = [14*x - 520 for x in X]
h4 = [16*x - 530 for x in X]
h5 = [18*x - 540 for x in X]
h6 = [20*x - 550 for x in X]
h7 = [22*x - 550 for x in X]
h8 = [24*x - 550 for x in X]

plt.plot(X, h1)
plt.plot(X, h2)
plt.plot(X, h3)
plt.plot(X, h4)
plt.plot(X, h5)
plt.plot(X, h6)
plt.plot(X, h7)
plt.plot(X, h8)

# Graph axes and title
plt.ylabel("Dollars")
plt.xlabel("Farenheit")
plt.title("Hypothesis Functions")

# Plot Hypothesis Functions
plt.show()
```

Out of all of the hypothesis functions plotted above, which is closest to the true linear regression line? How do you know? How can you show that one of these lines (and by extension its $\theta$ parameters) is closer to the optimal $\beta$ values?

# Cost Functions

Functions that gradient descent is trying to minimize are usually referred to as "cost functions" these functions are also referred to by names like: loss function, error function, or scoring function.

We can measure the quality of our hypotheses using a **cost function**. A cost function compares our hypothesis line to our actual y values and gives it a score based on how well our hypothesis function fits the data. How might we define a cost function to give a score to how well our hypotheses fit the data? Well, you may remember this little beauty:

\begin{align}
SSE = \sum_{i=1}^{n}(y_i - \hat{y})^2
\end{align}

This is the equation for finding the sum of the squared error (or residuals) between our fitted values and our datapoints. To remind you what this looks like here's an old graph that you might be familiar with.

```python
# Fit Model
model = LinearRegression()
model.fit(X, y)

# Assign coefficient and intercept to variables (beta_1, and beta_0)
beta_1 = model.coef_[0]
beta_0 = model.intercept_

print(beta_1)
print(beta_0)

# Make a single prediction for 55 degrees farenheit
x_value = 55

# Calculate predicted value
prediction = beta_1*x_value + beta_0

# Creat Scatterplot
plt.scatter(df.Farenheit, df.Dollars)

# Plot the Regression Line and predicted value
y_hat = [beta_1*x + beta_0 for x in X]
plt.plot(X, y_hat)
plt.plot(x_value, prediction, 'ro')

# Graph axes and title
plt.ylabel("Dollars")
plt.xlabel("Farenheit")
plt.title("Ice Cream Sales and Temperature")

# Plot the error (residual)
plt.arrow(55.2, 600, 0, 150, color='black')
plt.show()
```

This is a graph showing our Linear Algebra derived optimal coefficients for this problem with the black line representing a "residual" or "error" or in other words, the vertical distance between a predicted point and an actual data point.

What would these errors look like for our hypothesis lines? Would the errors be big or small? This was probably already clear to you but this is simply a way of formalizing how we "minimize the sum of the squared residuals"

We might rewrite our SSE equation in terms of our hypothesis functions instead of our already optimized function like so:

\begin{align}
SSE = \sum_{i=1}^{n}(y_i - h(x_i))^2
\end{align}

The goal is to find the values for $\theta_0$ and $\theta_1$ (contained within $h(x)$) that minimize this cost function. What's good at finding the minimum of things? Gradient Descent!

## What does this cost function look like?

```python
# Define a function that can calculate SSE for any hypothesis function
# print(X)
# print(y)

def SSE(theta_0, theta_1, y, X):
  squared_residuals = []
  for index, value in enumerate(y):
    h_x = theta_0 + theta_1*X[index][0]
    residual = h_x - value
    squared_residual = residual**2
    squared_residuals.append(squared_residual)
  return sum(squared_residuals)

SSE_1 = SSE(-600, 17, y, X)
SSE_2 = SSE(-600, 19, y, X)
SSE_3 = SSE(-600, 21, y, X)
SSE_4 = SSE(-600, 23, y, X)
SSE_5 = SSE(-600, 25, y, X)
SSE_6 = SSE(-600, 27, y, X)
SSE_7 = SSE(-600, 29, y, X)
SSE_8 = SSE(-600, 31, y, X)
SSE_9 = SSE(-600, 33, y, X)

print(SSE_1)
print(SSE_2)
print(SSE_3)
print(SSE_4)
print(SSE_5)
print(SSE_6)
print(SSE_7)
print(SSE_8)
print(SSE_9)

plt.scatter([17,19,21,23,25,27,29,31,33], [SSE_1, SSE_2, SSE_3, SSE_4, SSE_5, SSE_6, SSE_7, SSE_8, SSE_9])
plt.title("Cost Function h(x) holding theta_0 fixed")
plt.xlabel('theta_1')
plt.ylabel('SSE')
```

What shape does our cost function have when we only vary the slope variable and hold the other constant? It has the shape of a parabola.

If we varied both values we would then have a mapping of a higher dimension so what do you think our cost function would look like if we varied both $\theta_0$ and $\theta_1$?

Well, it turns out that it would look something like this:

Where the x axis is $\theta_0$ the z axis is $\theta_1$ and the y axis is our $SSE(\theta_0, \theta_1)$

# Blindfolded Hiker on our Linear Regression Cost Function

What if we put our blindfolded hiker onto our new cost functions and told him to get to the bottom? What might his travels look like? Maybe they would look something like this:

If we took our 3D plot and turned it into a contour plot his travels might look something like this:

# Using Gradient Descent to Minimize Cost Functions

We'll first talk about using Gradient Descent to minimize an arbitrary cost function and then later we'll implement it specifically on our linear regression example.

First lets talk about the equation for gradient descent:

- $J$ represents our cost function.
- The derivative term in front of $J$ (the fraction part) is our "gradient" in n-dimensional space or our "slope" in 2D space.
- $\alpha$ is our "Learning Rate" (how big of steps that we we initially take)
- $\theta_j$ represents our current position. So all of the RHS (right hand side) of this equation is calculated all at once and is then used to replace our current position with the new updated position.

1) Start off with initial guesses for $\theta_0$ and $\theta_1$. (We'll just start them both out at 0)

2) Change $\theta_0$ and $theta_1$ a little bit (according to the above equation) and reevaluate our cost function.

But first, lets demonstrate this process on a simple $X^2$ graph. This kind of a graph could come from a cost function that only has one parameter (we demonstrated this above when we held our intercept value fixed and only changed the slope value)

```python
# Plot y=x^2 function
import matplotlib.pyplot as plt

a=[]
b=[]

for x in range(-50,50,1):
    y=x**2+2*x+2
    a.append(x)
    b.append(y)

fig= plt.figure()
axes=fig.add_subplot(111)
axes.plot(a,b)

# Plot Gradient Descent Equation Demonstration
plt.plot(-40, 1500, 'ro')
plt.arrow(-40, 1500, 10, -680, linewidth=5, color='yellow')
plt.plot(-30, 820, 'ro')
plt.arrow(-30, 820, 8, -380, linewidth=5, color='yellow')
plt.plot(-22, 440, 'ro')
plt.arrow(-22, 440, 6, -220, linewidth=5, color='yellow')
plt.plot(-16, 220, 'ro')
plt.arrow(-16, 220, 5, -120, linewidth=5, color='yellow')
plt.plot(-11, 100, 'ro')
plt.arrow(-11, 100, 4, -70, linewidth=5, color='yellow')
plt.plot(-6, 30, 'ro')
plt.arrow(-6, 30, 3, -20, linewidth=5, color='yellow')
plt.plot(-3, 10, 'ro')
plt.arrow(-3, 10, 1, 0, linewidth=5, color='yellow')
plt.plot(-.5, 0, 'ro')

plt.show()
```

We'll move a copy of our function down here for convenience

Notice that because our step size (alpha) is multiplied by our gradient (slope) that even though we initialize our step size with a constant value it become altered by the value of the gradient so that we take smaller and smaller steps as the graph gets flatter. (closer to a minimum hopefully).

# Optimizing The Learning Rate

This is part of a process that we call hyperparameter tuning. Hyperparameter tuning in its most crude form basically means try out different features of your parameters (like learning rate) and see what happens to your results. Keep on iterating until you converge on an optimal learning rate. (Maybe we can implement gradient descent to find the most optimal learning rate for our gradient descent. 🤔)

You can read up on different techniques for choosing an appropriate learning rate. We will explore more sophisticated Hyperparameter Tuning techniques later on in the class.

You'll usually be fine with a pretty small learning rate (.01 or .02) when you're solving linear regression using Gradient Descent (as long as you use a sufficient number of iterations)

# Gradient Descent Implementation for Bivariate Linear Regression

## Standardize X matrix & look at OLS coefficients for comparison

```python
## Lets implement it and see how close GD comes to sklearn!

# Lets Implement it!!!

# First we'll calculate our parameters using sklearn to see how close our
# Gradient Descent Method gets.

# Read Data
df = pd.read_csv('https://raw.githubusercontent.com/ryanleeallred/datasets/master/Ice_Cream_Sales.csv')

# Create X matrix and y vector
X = df['Farenheit'].values[:, np.newaxis]

# If we don't do this our GD stuff will break
X = (X - X.mean()) / X.std()

# Matrix version makes cool linalg tricks possible
X_linalg = np.c_[np.ones(X.shape[0]), X]

print(X_linalg)

# Boring y variable list
y = df['Dollars'].values

# Fit Model
model = LinearRegression()
model.fit(X, y)

# Assign coefficient and intercept to variables (beta_1, and beta_0)
beta_1 = model.coef_[0]
beta_0 = model.intercept_

print("beta_1: ", beta_1)
print("beta_0: ", beta_0)
```

    [[ 1.         -1.61957156]
     [ 1.         -1.49352   ]
     [ 1.         -0.98931376]
     [ 1.         -0.65317627]
     [ 1.          0.10313309]
     [ 1.          0.31321903]
     [ 1.          0.43927059]
     [ 1.          0.64935652]
     [ 1.          0.77540808]
     [ 1.          0.81742527]
     [ 1.          1.65776901]]
    beta_1:  587.5808413073931
    beta_0:  1145.4545454545455

```python
# Plot the points
plt.scatter(X,y)

# Plot the line of best fit
y_hat = [beta_1*x + beta_0 for x in X]
plt.plot(X,y_hat)

# Add axes and title
plt.ylabel("Dollars")
plt.xlabel("Standardized Farenheit")
plt.title("Ice Cream Sales and Temperature")
plt.show()
```

## Set initial hyperparameters

```python
# Hyperparameters
alpha = 0.02 # Step Size
iterations = 2000 # Number of iterations
theta = [0, 0]

# Other Important Variables
n = y.size
np.random.seed(42)
```

## Calculate Predictions

```python
prediction = np.dot(X_linalg, theta)
print(prediction)
```

    [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]

## Calculate Error

```python
print('y: \n', y)
error = prediction - y
print('\n error \n:', error)
```

    y:
     [ 200  400  500  600 1100 1200 1500 1700 1800 1600 2000]

     error
    : [ -200.  -400.  -500.  -600. -1100. -1200. -1500. -1700. -1800. -1600.
     -2000.]

## Update Theta List Values

```python
# print(np.dot(X_linalg.T, error))

print(X_linalg.T)
print('\n')
print(error)
print('\n')
print(alpha*(1/n)*np.dot(X_linalg.T, error))

theta = theta - (alpha * (1/n)*np.dot(X_linalg.T, error))

print(theta)
```

    [[ 1.          1.          1.          1.          1.          1.
       1.          1.          1.          1.          1.        ]
     [-1.61957156 -1.49352    -0.98931376 -0.65317627  0.10313309  0.31321903
       0.43927059  0.64935652  0.77540808  0.81742527  1.65776901]]

    [ -196.12349354  -394.6421839   -488.71694534  -584.76678631
     -1075.87892848 -1173.41007908 -1471.92876944 -1669.45992004
     -1767.97861041 -1567.48484053 -1957.60944294]

    [-22.45090909 -11.51658449]
    [45.36       23.26820132]

## Minimal Gradient-Descent Implementation

```python
# Hyperparameters
alpha = 0.02 # Step Size
iterations = 2000 # Number of iterations
theta = [0, 0]

# Other Important Variables
n = y.size
np.random.seed(42)

def gradient_descent(x, y, theta, iterations, alpha):
  for i in range(iterations):
    prediction = np.dot(X_linalg, theta)
    error = prediction - y
    theta = theta - (alpha * (1/n)*np.dot(X_linalg.T, error))
  return theta

final_theta = gradient_descent(X_linalg, y, theta, iterations, alpha)
print("Gradient Descent Results: {:.9f}, {:.14f}".format(final_theta[0], final_theta[1]))
```

## Implementation with saved cost and theta values for graphing costs and animations

```python
# Hyperparameters
alpha = .02 # Step Size
iterations = 2000 # Number of iterations
theta = [0, 0]

# Other Important Variables
n = y.size
np.random.seed(42)

def gradient_descent(x, y, theta, iterations, alpha):
  past_costs = []
  for i in range(iterations):
    prediction = np.dot(X_linalg, theta)
    error = prediction - y
    theta = theta - (alpha * (1/n)*np.dot(X_linalg.T, error))
    cost = np.dot(error.T, error)
    past_costs.append(cost)
  return theta, past_costs

final_theta, past_costs = gradient_descent(X_linalg, y, theta, iterations, alpha)
print("Gradient Descent Results: {:.9f}, {:.14f}".format(final_theta[0], final_theta[1]))
```

    Gradient Descent Results: 1145.454545455, 587.58084130739041

## Cost by iterations

```python
plt.plot(past_costs)
plt.title('Cost Reduction by Iteration')
plt.xlabel('No. of iterations')
plt.ylabel('Cost')
plt.show()
```

```python

```

## Animation Code

```python
import matplotlib.animation as animation

#Set the plot up,
fig = plt.figure()
ax = plt.axes()
plt.title('Ice Cream Sales vs Temperature')
plt.xlabel('Temperature in Farenheit')
plt.ylabel('Revenue ($)')
plt.scatter(X_linalg[:,1], y, color='red')
line, = ax.plot([], [], lw=2)
annotation = ax.text(-1, 700000, '')
annotation.set_animated(True)
plt.close()

#Generate the animation data,
def init():
    line.set_data([], [])
    annotation.set_text('')
    return line, annotation

# animation function.  This is called sequentially
def animate(i):
    x = np.linspace(-5, 20, 1000)
    y = past_thetas[i][1]*X_linalg + past_thetas[i][0]
    line.set_data(x, y)
    annotation.set_text('Cost = %.2f e10' % (past_costs[i]/10000000000))
    return line, annotation

anim = animation.FuncAnimation(fig, animate, init_func=init,
                               frames=300, interval=0, blit=True)

anim.save('animation.gif', writer='imagemagick', fps=30)
```

## Display Animation

This code probably won't work well unless you run it in a local ipython notebook. And maybe not then either. 😉

```python
#Display the animation...
import io
import base64
from IPython.display import HTML

filename = 'animation.gif'

video = io.open(filename, 'r+b').read()
encoded = base64.b64encode(video)
HTML(data=''''''.format(encoded.decode('ascii')))
```

## Gradient Descent *is* Machine Learning

What does Gradient Descent do? It finds model parameters that minimize a cost function. (That get the best results!) This cuts to the very heart of what machine learning is: Solving uniquely specified optimization problems. I hope you can see how this iterative process of seeing an optimum is not just good for machines, but can be an elegant way of describing learning in general. For both humans and computers!

## Improvements to Gradient Descent

There are **many** Gradient-Descent based optimizers out there that seek improve upon the "vanilla" gradient descent that we have implemented here.

What if we had a cost function that wasn't convex? What if our cost function had multiple local minima?

But what if we had initialized our theta values just slightly differently. Where would the gradient have taken us?

Hopefully these two images help you see how with more complex cost functions it can be difficult to find the global minimum and even slightly different initial values can lead to completely different results. The question about how to overcome these challenges has lead to a plethora of gradient-descent based optimizers all seeking to overcome these challenges.

If you feel like you have a good grasp of what I'm calling "vanilla" gradient descent then start delving into understanding these different optimizers:

[Overview of GD-based optimizers](http://ruder.io/optimizing-gradient-descent/)

[Siraj Raval - Evolution of Gradient Descent-Based Optimizers](https://youtu.be/nhqo0u1a6fw)