# Posterior Approximation

## The Law of Large Numbers and Monte Carlo Estimation

What if there's no well-defined PDF for our product of likehood and prior(s)? It means we cannot use `scipy.stats.norm`, and its convenient `mean()`, `pdf()`, `cdf()`, or `ppf()` functions to establish probabilistic answers to questions about the posterior. Is it possible to answer such questions in a different way? Assuming we have a sample $y_1,\ldots,y_n$ from an _unknown_ distribution:

```python
import numpy as np

y = np.random.normal(loc=130, scale=10, size=10)  # although using samples from a normal distribution, pretend to not know the true distribution
```

The distribution is not defined by a PDF, but by a **sample**. According to the [Central Limit Theorem](https://en.wikipedia.org/wiki/Central_limit_theorem), it is possible to approximate the distribution mean with the sample mean, with large enough sample. Is $n=10$ good enough?

```python
np.mean(y)
```

```python
np.std(y)
```

As [mentioned before](./notebooks/01_bayes_rule_intro.ipynb#Continuous:), the probability of $Y$ being between e.g. 110 and 130 is

$$Pr(110 \le Y \le 130) = \int_{110}^{130}p(y)$$

However, $p(y)$ is undefined and the only thing we have is a sample. But the sample can be used to approximate the integral:

```python
def sample_cdf(x, sample):
    return (sample <= x).sum() / sample.size  # "integrating" by counting
```

```python
sample_cdf(130, y) - sample_cdf(110, y)
```

This can be compared with the probability obtained from the true normal distribution:

```python
from scipy.stats import norm

unknown_norm = norm(loc=130, scale=10)
unknown_norm.cdf(130) - unknown_norm.cdf(110)
```

Samples also allow us to compute quantiles:

```python
def sample_ppf(p, sample):
    p_index = int(np.round((sample.size - 1) * p))
    return np.sort(sample)[p_index]
```

So the range of $y$ for which the probability is 80% that it contains $y$'s true mean is:

```python
f'[{sample_ppf(0.1, y)} - {sample_ppf(0.9, y)}]'
```

where the interval from the true normal distribution would be:

```python
f'[{unknown_norm.ppf(0.1)} - {unknown_norm.ppf(0.9)}]'
```

Using a larger sample leads to more accurate estimations:

```python
y_large = np.random.normal(loc=130, scale=10, size=10000)

{
    'mean (130)': np.mean(y_large),
    'std (10)': np.std(y_large),
    'p_110_130 (0.4772)': sample_cdf(130, y_large) - sample_cdf(110, y_large),
    '80% ([117.2 - 142.8])': f'[{sample_ppf(0.1, y_large)} - {sample_ppf(0.9, y_large)}]'
}

```

### Chaining Simulations for Hierarchical Models

This process of computing means, variances, probabilities, percentiles (or any other quantities of interest) from simulated samples is called **Monte Carlo Estimation**. When using more complex models, simulations such as above can also be **chained**. For example, consider a model similar to the one in the [previous note](./02_generative_models.ipynb#Example,-observing-a-single-value-with-a-simple-model):

$$
\begin{align}
\mu &\sim\color{red}{\mathcal{t}(m_0, s_0, \nu)}\,\mathrm{(prior)}\\
y &\sim\color{blue}{\mathcal{N}(\mu, \sigma^2_0)}\,\mathrm{(likelihood)}\\
m_0 &=130\\
s_0 &=10\\
\nu &=(n-1)\,\text{degrees of freedom for a sample size of}\,n\\
\sigma^2_0 &=100
\end{align}
$$

The joint density $p(y, \mu) = \color{blue}{p(y\mid\mu)}\color{red}{p(\mu)}$ cannot be expressed as a known family (e.g. Normal) of probability densities, but can be estimated by simulation:

```python
from scipy.stats import t

# basing the prior on a sample of n=10, simulating m=1000 samples
mu = t.rvs(df=9, loc=130, scale=10, size=1000)

# an array of n location parameters can be plugged in to get n new samples
y_chained = norm.rvs(loc=mu, scale=10)

{
    'mean (130)': np.mean(y_chained),
    'std (10)': np.std(y_chained),
    'p_110_130 (0.4772)': sample_cdf(130, y_chained) - sample_cdf(110, y_chained),
    '80% ([117.2 - 142.8])': f'[{sample_ppf(0.1, y_chained)} - {sample_ppf(0.9, y_chained)}]'
}
```

These are samples from the **prior predictive** distribution, i.e. given the likelihood and all the priors in a model, what values of $y$ can we expect (before having observed any data)?

- Can we use the above approach to sample from the _posterior_ distribution $p(\mu\mid y)$? Why (not)?

## Markov Chains

A [**Markov Chain**](https://en.wikipedia.org/wiki/Markov_chain) is a sequence of random variables $X_1,\ldots,X_n$, with $1,\ldots,n$ representing points in time, where the probability (density) of $X_{t+1}$ only depends on the value $X_t$, i.e.

$$
p(X_{t+1}\mid X_t,X_{t-1},\ldots,X_2,X_1) = p(X_{t+1}\mid X_t)
$$

A random walk with $X_0\sim\mathcal{N}(0, 1)$ and $X_{t+1}\mid X_t\sim\mathcal{N}(X_t, 1)$ is an example of a Markov Chain:

```python
def normal_random_walk(sigma, mu_init=0):
    mu = mu_init

    while True:
        yield mu
        mu = np.random.normal(loc=mu, scale=sigma)
```

```python
import plotly.graph_objs as go

random_walk_iter = normal_random_walk(1)
n=1000

random_walk_trace = go.FigureWidget(
    data=[go.Scatter(x=list(range(n)), y=[next(random_walk_iter) for _ in range(n)], showlegend=False)]
)

random_walk_trace
```

What happens with multiple random walks?

```python
import pandas as pd

trace_df = pd.DataFrame()

for i in range(10):
    random_walk_iter = normal_random_walk(1)  # re-initialize generator for every new random walk
    trace = [next(random_walk_iter) for _ in range(n)]
    trace_df[f'trace_{i}'] = trace
    random_walk_trace.add_scatter(x=list(range(n)), y=trace, showlegend=False)
```

They look ... quite random:

```python
trace_df.describe()
```

However, with a little tweak of the random walk...

```python
def tweaked_random_walk(sigma, mu_init=0, phi=0.9):
    mu = mu_init

    while True:
        yield mu
        mu = np.random.normal(loc=phi*mu, scale=sigma)
```

```python
n=1000
sigma=20

random_walk_iter = tweaked_random_walk(1, mu_init=np.random.normal(loc=0, scale=sigma))

random_walk_trace = go.FigureWidget(
    data=[go.Scatter(x=list(range(n)), y=[next(random_walk_iter) for _ in range(n)], showlegend=False)]
)

random_walk_trace
```

```python
trace_df = pd.DataFrame()

for i in range(10):
    random_walk_iter = tweaked_random_walk(1, mu_init=np.random.normal(loc=0, scale=sigma))  # re-initialize generator for every new random walk
    trace = [next(random_walk_iter) for _ in range(n)]
    trace_df[f'trace_{i}'] = trace
    random_walk_trace.add_scatter(x=list(range(n)), y=trace, showlegend=False)

trace_df.describe()
```

... the traces converge to a **stationary distribution** (with mean 0). When experimenting with different values for $\phi$ in the `tweaked_random_walk()` generator, it turns out that the random walks converge in the case of $-1\lt\phi\lt1$. In that case, the generator `tweaked_random_walk()` can be considered a **sampler** for $\mathcal{N}(0, \frac{1}{1-\phi^2})$.

**Note:** Markov Chains are not limited to continuous variables. It is possible to create discrete Markov Chains by modeling probabilistic state transitions between discrete variables.

Would it be possible to construct such a sampler for arbitrary complex distributions? Such as a posterior $p(\theta\mid y)$?

## Getting Picky - The Metropolis Sampler

Assume a model that estimates the month-over-month _change_ in mean exercise heart rate $\mu$ for a population. Such a model can be defined as:

$$
\begin{align}
y &\sim\color{blue}{\mathcal{N}(\mu, 1)}\,\mathrm{(likelihood)}\\
\mu &\sim\color{red}{\mathcal{t}(0, 1, 1)}\,\mathrm{(prior)}\\
\end{align}
$$

with the prior for $\mu$ coming from a standard $\mathcal{t}$ distribution, i.e. centered around 0 with scale equal to 1 standard deviation. This allows to write the posterior as:

$$
\begin{align}
p(\mu\mid y_1,\ldots,y_n) &\propto \color{blue}{p(y_1,\ldots,y_n\mid\mu)}\color{red}{p(\mu)}\\
&= \color{blue}{\prod_{i=1}^n \frac{1}{\sqrt{2\pi}}exp\left\{-\frac{1}{2}{(\color{black}{y_i} - \mu)}^2\right\}}\color{red}{\frac{1}{\pi(1+\mu^2)}}\\
&\propto\ldots\\
&\propto \frac{exp\left[n(\bar{y}\mu - \frac{\mu^2}{2})\right]}{1 + \mu^2}\,= g(\mu)
\end{align}
$$

where $n$ and $\bar{y}$ are known and fixed. To summarize, $g(\mu)$ is:

- a PDF,
- proportional to $p(\mu\mid y)$

The [Metropolis algorithm](https://en.wikipedia.org/wiki/Metropolis%E2%80%93Hastings_algorithm) allows to sample from the posterior $p(\mu\mid y)$ by creating a Markov Chain with a stationary distribution that approximates the posterior. It does so by iteratively accepting or rejecting samples drawn from another distribution that is easier to sample.

Remember this?

```python
norm.pdf(2) / norm.pdf(1)
```

$$
\begin{align}
p(\mu\mid y) &\propto g(\mu) \Leftrightarrow\\
\frac{p(\mu_1\mid y)}{p(\mu_2\mid y)} &= \frac{g(\mu_1)}{g(\mu_2)}\\
\text{or}\\
\frac{p(\mu_{t+1}\mid y)}{p(\mu_t\mid y)} &= \frac{g(\mu_{t+1})}{g(\mu_t)}\\
\end{align}
$$

Since we can evaluate $g(\mu)$, the ratio $\frac{g(\mu_1)}{g(\mu_2)}$ tells us how much more/less likely it is to see $\mu_1$ compared to $\mu_2$ in the **target distribution** $p(\mu\mid y)$! In the Metropolis sampler, this ratio is used to accept or reject new samples $\mu_{t+1}$ based on the previous sample $\mu_t$. This approach can be integrated into the random-walk sampler:

```python
def log_g(y_bar, n, mu):
    """
    Returns the joint density of y and mu assuming a normal likelihood and t prior,
    at log scale (to prevent under/overflow issues)
    """
    return n * (y_bar * mu - mu**2 / 2) - np.log(1 + mu**2)
```

```python
def metropolis_sampler(y, mu_init=0.0, sigma=1):
    samples = 0  # numer of samples drawn so far
    accept = 0  # number of accepted samples so far
    mu = mu_init  # current state of sampler
    y_bar = np.mean(y)
    n = y.size

    while True:
        yield (mu, accept/samples if samples > 0 else 0)

        candidate_mu = np.random.normal(loc=mu, scale=sigma)
        acceptance_ratio = np.exp(log_g(y_bar, n, candidate_mu) - log_g(y_bar, n, mu))

        if np.random.uniform() < acceptance_ratio:
            # accept the new candidate
            mu = candidate_mu
            accept += 1

        samples += 1

```

Now, assume we observe the following month-over-month changes of mean exercise heart rate for 10 athletes:

```python
y = np.array([3, 0, -2, 5, 1, 0, 8, -4, -1, 3])
np.mean(y)
```

```python
sampler = metropolis_sampler(y, sigma=5)
samples = 1000

large_step_trace = [next(sampler) for _ in range(samples)]
np.mean([sample[0] for sample in large_step_trace])
```

```python
metropolis_trace = go.FigureWidget(
    data=[go.Scatter(x=list(range(samples)), y=[sample[0] for sample in large_step_trace], name=f'large steps, acceptance rate = {large_step_trace[-1][1]:.3f}')]
)

metropolis_trace
```

```python
sampler = metropolis_sampler(y, sigma=0.05)
small_step_trace = [next(sampler) for _ in range(samples)]
metropolis_trace.add_scatter(x=list(range(samples)), y=[sample[0] for sample in small_step_trace], name=f'small steps, acceptance rate = {small_step_trace[-1][1]:.3f}');
np.mean([sample[0] for sample in small_step_trace])
```

```python
sampler = metropolis_sampler(y, sigma=1.2)
good_step_trace = [next(sampler) for _ in range(samples)]
metropolis_trace.add_scatter(x=list(range(samples)), y=[sample[0] for sample in good_step_trace], name=f'good step size, acceptance rate = {good_step_trace[-1][1]:.3f}');
np.mean([sample[0] for sample in good_step_trace])
```

- Why is the posterior mean for $\mu$ smaller than the sample mean $\bar{y}$?

Finally, let's take a look at the posterior distribution for $\mu$:

```python
go.FigureWidget(
    data=[go.Histogram(x=[sample[0] for sample in good_step_trace])]
)
```


# Non Linear Controller

```python
%matplotlib inline
%config InlineBackend.figure_format = 'retina'

%load_ext autoreload
%autoreload 2

import numpy as np
import math
from math import sin, cos
import matplotlib.pyplot as plt
import matplotlib.pylab as pylab

from drone import Drone2D

import trajectories
import simulate
import plotting

pylab.rcParams['figure.figsize'] = 10, 10
```

#### TODO - Implement 2D controller with NON-LINEAR control equations.

The non-linear equations of motion are shown below.

$$\begin{align}
\ddot{z} &= g - \frac{u_1}{m}\cos\phi
\\
\\
\ddot{y} &= \frac{u_1}{m}\sin\phi
\\
\\
\ddot{\phi} &= \frac{u_2}{I_x}
\end{align}$$

Recalling that $$ \bar{u_1} ==\ddot{z} ; \bar{u_2}  == \ddot{\phi} ;  \phi_{\text{command}} == \phi $$
$$
$$
Then these equations can be solved for $u_1$, $\phi_{\text{command}}$, and $u_2$.

$$\begin{align}
u_1 &= \frac{m(g - \bar{u_1})}{\cos\phi}
\\
\\
\phi_{\text{command}} &= \sin^{-1}\left(\frac{m\ddot{y}_{\text{target}}}{u_1}\right)
\\
\\
u_2 &= I_x \bar{u}_2
\end{align}$$

The first equation will be useful when implementing `altitude_controller`, the second when implementing `lateral_controller`, and the third when implementing `attitude_controller`.

Note that $\ddot{y}_{\text{target}}$ is like $\bar{u}_1$ or $\bar{u}_2$. It comes from PD control on the controller's inputs.

```python
class NonLinearCascadingController:

    def __init__(self,
                 m,
                 I_x,
                 z_k_p=1.0,
                 z_k_d=1.0,
                 y_k_p=1.0,
                 y_k_d=1.0,
                 phi_k_p=1.0,
                 phi_k_d=1.0):

        self.z_k_p = z_k_p
        self.z_k_d = z_k_d
        self.y_k_p = y_k_p
        self.y_k_d = y_k_d
        self.phi_k_p = phi_k_p
        self.phi_k_d = phi_k_d

        self.g = 9.81
        self.I_x = I_x
        self.m = m

    def altitude_controller(self,
                    z_target,
                    z_actual,
                    z_dot_target,
                    z_dot_actual,
                    z_dot_dot_target,
                    phi_actual):

        # TODO (recommended to do AFTER attitude)
        #   Implement feedforward PD control to calculate
        #   u_1_bar and then use the non-linear math from above
        #   to transform u_1_bar into u_1 and then return u_1

        # for reference this is the linear implementation:
#         z_err = z_target - z_actual
#         z_err_dot = z_dot_target - z_dot_actual

#         p_term = self.z_k_p * z_err
#         d_term = self.z_k_d * z_err_dot

#         u_1_bar = p_term + d_term + z_dot_dot_target
#         u_1 = self.m* (self.g - u_1_bar)
        u_1_bar = self.z_k_p * (z_target - z_actual) \
            + self.z_k_d * (z_dot_target - z_dot_actual) \
            + z_dot_dot_target

        u_1 = self.m * (self.g - u_1_bar) / math.cos(phi_actual)
        return u_1

    def lateral_controller(self,
                        y_target,
                        y_actual,
                        y_dot_target,
                        y_dot_actual,
                        u_1,
                        y_dot_dot_ff=0.0,
                        ):

        # TODO (recommended to do AFTER attitude)
        #   Implement feedforward PD control to calculate
        #   y_dot_dot_target and then use the non-linear math from above
        #   to transform y_dot_dot_target into phi_commanded
        #   and then return phi_commanded

        #for reference this is the linear implementation:
#         y_err = y_target - y_actual
#         y_err_dot = y_dot_target - y_dot_actual

#         p_term = self.y_k_p * y_err
#         d_term = self.y_k_d * y_err_dot

#         y_dot_dot_target = p_term + d_term + y_dot_dot_ff
#         phi_commanded = y_dot_dot_target / self.g

        y_dot_dot_target = self.y_k_p * (y_target - y_actual) \
            + self.y_k_d * (y_dot_target - y_dot_actual) \
            + y_dot_dot_ff
        phi_commanded = math.asin(self.m * y_dot_dot_target) / u_1
        return phi_commanded

    def attitude_controller(self,
                        phi_target,
                        phi_actual,
                        phi_dot_actual,
                        phi_dot_target=0.0
                        ):
        # TODO (recommended to do FIRST)
        #   Implement PD control to calculate u_2_bar
        #   and then use the linear math from above to
        #   transform u_2_bar into u_2 and then return u_2

        # for reference this is the linear implementation:
#         phi_err = phi_target - phi_actual
#         phi_err_dot = phi_dot_target - phi_dot_actual

#         p_term = self.phi_k_p * phi_err
#         d_term = self.phi_k_d * phi_err_dot

#         u_2_bar = p_term + d_term
#         u_2 = (u_2_bar) * self.I_x

        u_2_bar = self.phi_k_p * (phi_target - phi_actual) \
            + self.phi_k_d * (phi_dot_target - phi_dot_actual)
        u_2 = self.I_x * u_2_bar
        return u_2
```

The flight path we'll use to test our controller is a figure 8 described as follows:
$$
\begin{align}
z & = a_z \sin{\omega_z t} \\
y & = a_y \cos{\omega_y t}
\end{align}
$$

where $\omega_y = \omega_z / 2$.

**NOTE on tuning:**

A good approach is to start with the inner loop, the ATTITUDE controller parameters (phi_k_p, phi_k_d) and the move forward with the LATERAL and ALTITUDE controller.
Also, the instruction for the "linear" implementation can be useful here.

```python
# TESTING CELL
#
# Note - this cell will only give nice output when your code
#  is working AND you've tuned your parameters correctly.
#  you might find it helpful to come up with a strategy
#  for first testing the inner loop controller and THEN
#  testing the outer loop.
#
# Run this cell when you think your controller is ready!
#
# You'll have to tune the controller gains to get good results.

#### CONTROLLER GAINS (TUNE THESE) ######

z_k_p   = 20.0
z_k_d   = 2.0
y_k_p   = 2.5
y_k_d   = 0.1
phi_k_p = 55.0
phi_k_d = 2.0

#########################################

drone = Drone2D()

# INSTANTIATE CONTROLLER
non_linear_controller = NonLinearCascadingController(
    drone.m,
    drone.I_x,
    z_k_p=z_k_p,
    z_k_d=z_k_d,
    y_k_p=y_k_p,
    y_k_d=y_k_d,
    phi_k_p=phi_k_p,
    phi_k_d=phi_k_d
)

# TRAJECTORY PARAMETERS (you don't need to change these)
total_time = 30.0
omega_z = 1.0       # angular frequency of figure 8

# GENERATE FIGURE 8
z_traj, y_traj, t = trajectories.figure_8(omega_z, total_time, dt=0.02)
z_path, z_dot_path, z_dot_dot_path = z_traj
y_path, y_dot_path, y_dot_dot_path = y_traj

# SIMULATE MOTION
linear_history     = simulate.zy_flight(z_traj,
                                        y_traj,
                                        t,
                                        non_linear_controller,
                                        inner_loop_speed_up=10)
# PLOT RESULTS
pylab.rcParams['figure.figsize'] = 10,10
print(linear_history.shape)
plotting.plot_zy_flight_path(z_path, y_path, linear_history)
plotting.plot_z(linear_history, z_path, t)
plotting.plot_y(linear_history, y_path, t)
plotting.plot_phi(linear_history, t)
```

[Solution](/notebooks/Non-Linear%20Controller%20Solution.ipynb)


# Oscilador armónico amortiguado

Erwin Renzo Franco Diaz

```python
import sympy as sp
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
sp.init_printing()
```

Si no se ha dado alguna restricción a las cantidades simbólicas con las que se trabaja, $\texttt{sympy}$ buscará la solución más general a la ecuación de movimiento. Esto nos puede llevar a tener soluciones complejas o por casos. Para evitar esto, especificamos la naturaleza de nuestras cantidades. En este caso, el tiempo, el desplazamiento y las condiciones iniciales son reales, mientras que las constantes además, son positivas.

```python
t = sp.symbols('t', real = True)
x = sp.Function('x', real = True)(t)

w = sp.symbols('omega', positive = True)
g = sp.symbols('gamma', positive = True)

x0 = sp.symbols('x_0', real = True)
x0dot = sp.symbols('\dot{x_0}', real = True)

CI = {x.subs(t, 0): x0, sp.diff(x, t).subs(t, 0): x0dot}

eq_damped = sp.Eq(sp.diff(x, (t, 2)) + g*sp.diff(x, t) + w**2*x,  0)

eq_damped
```

## Caso1: $\gamma = 0.1(2\omega)$

```python
g1 = 0.1*2*w

eq1 = eq_damped.subs(g, g1)
sol1 = sp.dsolve(eq1, x, ics=CI)

sol1
```

## Caso 2: $\gamma = 0.5(2\omega)$

```python
g2 = 0.5*2*w

eq2 = eq_damped.subs(g, g2)
sol2 = sp.dsolve(eq2, x, ics=CI)

sol2
```

## Caso 3: $\gamma = 0.9(2\omega)$

```python
g3 = 0.9*2*w

eq3 = eq_damped.subs(g, g3)
sol3 = sp.dsolve(eq3, x, ics=CI)

sol3 = sol3.rewrite(sp.cos).simplify()
sol3
```

```python
w_num = 0.5
x0_num = 5
x0dot_num = 10

x_num1 = sp.lambdify(t, sol1.subs([(w, w_num), (x0, x0_num), (x0dot, x0dot_num)]).rhs, 'numpy')
x_num2 = sp.lambdify(t, sol2.subs([(w, w_num), (x0, x0_num), (x0dot, x0dot_num)]).rhs, 'numpy')
x_num3 = sp.lambdify(t, sol3.subs([(w, w_num), (x0, x0_num), (x0dot, x0dot_num)]).rhs, 'numpy')

t_plot = np.linspace(0, 50, 1000)
x_plot1 = x_num1(t_plot)
x_plot2 = x_num2(t_plot)
x_plot3 = x_num3(t_plot)

plt.plot(t_plot, x_plot1, t_plot, x_plot2, t_plot, x_plot3)
plt.legend(['$0.3(2\omega)$', '$0.5(2\omega)$', '$0.9(2\omega)$'])
```

# Dos péndulos acoplados

Indicando que los desplazamientos y las velocidades son reales, y que las constantes son positivas es posible resolver el sistema de ecuaciones de manera analítica.

```python
x1_c = sp.Function('x_1', real = True)(t)
x0_c = sp.Function('x_0', real = True)(t)
v = sp.Function('v', real = True)(t)
u = sp.Function('u', real = True)(t)

m, g, k, l = sp.symbols('m, g, k, l', positive = True)

x10, x00, v0, u0 = sp.symbols('x_1^{(0)}, x_2^{(0)}, v_0, u_0')

eq1 = sp.Eq(v, sp.diff(x1_c, t))
eq2 = sp.Eq(u, sp.diff(x0_c, t))
eq3 = sp.Eq(m*sp.diff(v, t) - m*(g/l)*x1_c - k*(x1_c - x0_c), 0)
eq4 = sp.Eq(m*sp.diff(u, t) - m*(g/l)*x0_c - k*(x0_c - x1_c), 0)

eqs = [eq1, eq2, eq3, eq4]

CI = {x1_c.subs(t, 0): x10, x0_c.subs(t, 0): x00, v.subs(t, 0): v0, u.subs(t, 0): u0}

x1_c, x0_c, v, u = sp.dsolve(eqs, [x1_c, x0_c, v, u], ics = CI)

x1_c = x1_c.rewrite(sp.cos).simplify()
x0_c = x0_c.rewrite(sp.cos).simplify()
```

La respuesta que da $\texttt{sympy}$ contiene una parte compleja con una constante arbitraria. Esto se debe a que se tiene una exponencial elevada a la raíz de un número negativo, el cual es complejo, por lo que junto con la $i$ en frente resulta, termina siendo real. Lo que se debe hacer entonces, es eliminar esta exponencial compleja usando la identidad de Euler

$$
e^{i\theta} = \cos{\theta} + i\sin{\theta}.
$$

```python
x1_c
```

```python
x0_c
```

```python
m_num_c = 0.5
g_num_c = 10
k_num_c = 0.5
l_num_c = 1

x10_num = 4
x00_num = -0.5
v0_num = 1
u0_num = 0

subst = [(m, m_num_c), (g, g_num_c), (k, k_num_c), (l, l_num_c), (x10, x10_num), (x00, x00_num), (v0, v0_num), (u0, u0_num)]

x1_num_c = sp.lambdify(t, x1_c.subs(subst).rhs, 'numpy')
x0_num_c = sp.lambdify(t, x0_c.subs(subst).rhs, 'numpy')

x1_plot_c = x1_num_c(t_plot)
x0_plot_c = x0_num_c(t_plot)

plt.plot(t_plot, x1_plot_c, t_plot, x0_plot_c)
plt.legend(['$x_1$', '$x_0$'])
plt.show()
```

```python
x0c
```


# Classification example

The goal of this script is to illustrate classification of timeseries as being
generated by observing stable or unstable objects.

For this, do the following:

    1. Definitions and imports
    2. Auxiliary functions
    3. Simulate the data and labels
    4. Feature extraction
    5. Training the SVM
    6. Plots and illustrations

The following entries may be changed to explore the script:

    * sigma_noise , sigma_signal  : Both together define the signal-to-noise ration in the data
    * corr_length_signal  : A quantity describing how slow the correlation between values drops
                            depending on temporal separation. Increase to generate a smoother signal.
    * n_expansion         : Determines, how many expansion coefficients are constructed for each time series.
                            The classifier is then trained on the expansion coefficients. Can be either 2 or 3.
    * n_episodes          : The number of time series used for training and testing
    * decision_rule       : The rule for deciding if a timeseries is associated to a failure by
                            assigning decision_rule(timeseries)=1 or decision_rule(timeseries)=0.

This script is given out as part of the Machine Learning Tutorial during IV2020, Munich. Please consult the slides for some background information regarding motivation and a possible physical setup for this toy example.

Written by Jemil Butt, Zan Gojcic, ETH Zurich.

  # 1. Definitions and imports  ------------------------------------------------

The support vector machine to be constructed in the codeblocks below is supposed to take as input timeseries $x(\cdot)$ and output a prediction if those timeseries are associated to the measurement of a stable object or an instable object (movement in positive direction). To learn the relationship

$$ \phi:X\ni x(\cdot)\mapsto \phi(x)=y \in \{0,1\}$$

between timeseries $x(\cdot)$ and instability as encoded by the binary quantity $y\in\{0,1\}$, labled sample data $\{(x_k,y_k)\}_{k=1}^n \in X\times Y$ is necessary. The way this sample data is generated can be controlled by specifying the smoothness of the underlying signal $(\rho_{signal}$) determining the stability and the signal-to-noise-ratio $(\sigma^2_{signal}/\sigma^2_{noise})$ determining the amount of completely non-indicative noise polluting whose superimposition with the signal forms the observations. The amount of labled timeseries data generated for training and test purposes can be adjusted by changing $n_{episodes}$.

The underlying rule for determining if a timeseries $x_k$ is associated to a stable $(y_k=0)$ or instable $(y_k=1)$ object is not known in practical applications and the dataset typically comes from compiling case examples to learn from. In this toy example we simulate the data and decide according to an automated underlying decision rule that is unknown to the classifier. It generates labels based on observed noisy timeseries $$x=x_{signal}+x_{noise}$$ by associating to $x$ the label "stable" or "instable" depending on $x_{signal}$ exceeding a certain threshold. The images below illustrate the type of the input data used to train the support vector machine.

Data episode 1 $\hspace{12cm}$ Data episode 2

Given only a sequence of noisy $x_k$'s and their corresponding labels $y_k$, the unknown rule for associating $y's$ to $x's$ is to be guessed and applied.

```

# 1.1 Import numerical and plotting libraries
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
import cvxopt
from sklearn.metrics import accuracy_score
import random
from mpl_toolkits.mplot3d.art3d import Poly3DCollection
from matplotlib.colors import ListedColormap

# 1.2 Define the number of observations and the time at which they take place
#       (Time interval is arbitrarily set to [0,1] to avoid scaling issues)
n_timesteps = 100
time = np.linspace(0, 1, n_timesteps)

#1.3 Parameters for simulationa and estimation

# Parameters for simulation               # The following parameters may be changed to explore the method
sigma_noise = 0.1                         # Quantifies the average magnitude of the noise
sigma_signal = 1                          # Quantifies the average magnitude of the signal
corr_length_signal = 1                    # A quantity describing how slow the correlation between values drops
                                          # depending on temporal separation.
n_expansion = 2                           # The number of polynomials in the parametric regression
n_episodes = 500                          # The number of simulations on which the decision rule is based
decision_rule=lambda x: x[-1] >= 1.0      # The rule for deciding if a failure occured or not. It is here
                                          # fixed as comparing the temporally last state to a threshold.
                                          # This is unknown to the algorithm.
                                          # The following is a decision rule to check for two-sided movement:
                                          # decision_rule=lambda x: x[-1] >= 1.0 or x[-1]<=-1

print('Packages imported. The parameters for simulation and ground truth generation have been defined and are now \
ready for later use.')

```

    Packages imported. The parameters for simulation and ground truth generation have been defined and are now ready for later use.

# 2. Auxiliary functions ----------------------------------------------------

Several special functions necessary for data management, embedding, and plotting are defined for later use. This section can be glossed over on first contact. Of course the code needs to be executed nonetheless!

```

# 2.1 Kernel-related functions

# Define the covariance functions of signal and noise for simulation
def cov_fun_signal(time_1, time_2, sigma_signal, corr_length):
    covariance = np.power(sigma_signal, 2) * np.exp(-np.power(((time_1 - time_2) / corr_length), 2))
    return covariance

# Define the kernel function for high dimensional embedding
def gaussian_kernel(x, y, sigma):
    return np.exp(- np.linalg.norm(x - y) ** 2 / (2 * sigma ** 2))

# Define a function that creates kernel matrices
def initialize_gram_matrix(X, sigma=0):
    n_samples, n_features = X.shape
    K = np.zeros((n_samples, n_samples))

    if not sigma:
        sigma = 1 / n_features

    for i, x_i in enumerate(X):
        for j, x_j in enumerate(X):
            K[i, j] = gaussian_kernel(x_i, x_j, sigma)
    return K

# 2.2 SVM-construction-related functions

# Given a solution to the Optimization problem in 4., calculate the intercept
# parameter of the svm
def calculate_intercept(lagr_mul, s_vector_labels, s_vectors):
    intercept = s_vector_labels[0]
    for i in range(len(lagr_mul)):
        intercept -= lagr_mul[i] * s_vector_labels[i] * gaussian_kernel(s_vectors[i],
                                                                        s_vectors[0], 1 / s_vectors.shape[1])
    return intercept

# Given a solution to the optimization problem in 4. and the intercept parameter,
# calculate the predicted class labels of all the observations accumulated in
# matrix X.
def predict(X, lagr_mul, s_vector_labels, s_vectors, intercept):
    y_pred = []
    # Iterate through list of samples and make predictions
    for sample in X:
        prediction = 0
        # Determine the label of the sample by the support vectors
        for cnt in range(len(lagr_mul)):
            prediction += lagr_mul[cnt] * s_vector_labels[
                    cnt] * gaussian_kernel(s_vectors[cnt], sample, 1 / s_vectors.shape[1])

        prediction += intercept
        y_pred.append(np.sign(prediction))
    return np.array(y_pred)

# 2.3 Data management functions

# Split data into a training set teaching the algorithm the prediction rules
# and a test set used for evaluating the prediction qualities.
def train_test_split(X, y, test_size=0.2,shuffle=False):
    combined_array = np.concatenate((X, y),axis=1)
    dim = combined_array.shape[1] - 1
    if shuffle:
        np.random.shuffle(combined_array)

    n_test = int(X.shape[0] * test_size)
    split = np.asarray(random.sample(range(X.shape[0]), n_test))

    return combined_array[~split,0:dim], combined_array[split,0:dim], combined_array[~split,dim], combined_array[split, dim]

# 2.4 2D and 3D plotting-related functions

# From a pair of vectors create two matrices of coordinates that correspond to the
# cartesian product of the coordinates such that xx enumerates all the x values
# in a rectangle of size x X y and yy enumerates all the y values in a rectangle
# of size x X y. This is for plotting a field of predictions with extent x in x-direction
# and extent y in y-direction.
def make_meshgrid(x, y, h=.02):
    x_min, x_max = x.min() - 1, x.max() + 1
    y_min, y_max = y.min() - 1, y.max() + 1
    xx, yy = np.meshgrid(np.arange(x_min, x_max, h),
                         np.arange(y_min, y_max, h))
    return xx, yy

# From a triple of vectors create three tensors of coordinates that correspond to the
# cartesian product of the coordinates such that xx, yy, and zz enumerate all the x,y,z values
# in a 3d Hyperrectangle of size x X y X z . This is for plotting a field of predictions
# with extent x in x-direction, extent y in y-direction, and extent z in z-direction.
def make_meshgrid3(x, y,z, h=.02):
    x_min, x_max = x.min() - 1, x.max() + 1
    y_min, y_max = y.min() - 1, y.max() + 1
    z_min, z_max = z.min() - 1, z.max() + 1
    xx, yy, zz = np.meshgrid(np.arange(x_min, x_max, h),
                         np.arange(y_min, y_max, h), np.arange(z_min, z_max, h))
    return xx, yy, zz

print('Auxiliary functions necessary for hyperplane construction and training successfully defined.')

```

    Auxiliary functions necessary for hyperplane construction and training successfully defined.

# 3. Simulate the data and labels -------------------------------------------

Simulate randomly chosen smooth signals by considering them to be drawn from a multivariate Gaussian distribution with dimension $n_{timesteps}$. With e.g. $n_{timesteps}=100$, each throw of the dice produces a vector of length 100 that we interpret as a function mapping dimension indices to function values. We then draw $x_{signal}$ according to
$$ x_{signal}\sim\mathcal{N}(0,K)$$
where $0\in \mathbb{R}^{100}$ and $K\in \mathbb{R}^{100 \times 100}$. $\mathcal{N}$ symbolizes the multivariate Gaussian distribution. Even though each draw is random and the different realizations are independent of each other, there exists significant spatial correlation between neighboring values.  More explanations of the intuition behind the simulation process may be found in the notes associated to the regression example. Each of the simulated timeseries is then subjected to noise so that the observations finally passed as input to the support vector machine satisfy $$x=x_{signal}+x_{noise}.$$

Some exemplary training data associated to negative and positive case examples can be seen in the image below.

Note that the decision rule classifying timeseries into the positive or negative class can be modified in section 1 if you want to alter it to something more sophisticated than just comparing the timeseries to a threshold.

```

# 3.1 Construct covariance matrices of signal - nontrivial, smooth correlation
Cov_mat_signal=np.zeros((n_timesteps,n_timesteps))
for i in range(n_timesteps):
    for j in range(n_timesteps):
        Cov_mat_signal[i,j]=cov_fun_signal(time[i],time[j],sigma_signal,corr_length_signal)

# 3.2 Construct covariance matrices of noise - uncorrelated, white noise
Cov_mat_noise =np.power(sigma_noise,2)*np.eye(n_timesteps)

# 3.3 Do the simulation
x_true = np.zeros((n_timesteps,n_episodes))
x_measured = np.zeros((n_timesteps,n_episodes))

for k in range(n_episodes):
    # First simulate random signal ...
    x_true[:,k]=np.random.multivariate_normal(np.zeros([n_timesteps]),Cov_mat_signal)

    # ... then add noise to create the synthetic measurements
    x_measured[:,k]=x_true[:,k]+np.random.multivariate_normal(np.zeros([n_timesteps]),Cov_mat_noise)

# 3.4 Generate the labels
labels_true=np.zeros((n_episodes,1))
for k in range(n_episodes):
    labels_true[k,0]=decision_rule(x_true[:,k])

print('Covariance matrices have been constructed. Simulations were carried out by sampling from \
multivariate normal distributions and employing the decision rule.')

```

    Covariance matrices have been constructed. Simulations were carried out by sampling from multivariate normal distributions and employing the decision rule.

# 4. Feature extraction -----------------------------------------------------

The features used to represent the timeseries $x$ are the regression coefficients $\alpha_1, \alpha_2$ of the best fit of a polynomial of degree $n_{expansion}=2$ to the timeseries $x_k$. To find these coefficients $\alpha$ solve the adjustment problem
$$\alpha^*=\underset{\theta\in \Theta}{\operatorname{argmin}}\|A\alpha-b\|^2_2$$
where $\alpha=(\alpha_1,\alpha_2)$, $b$ is the data $\| \cdot\|_2$ is the 2-norm indicating sums-of-squares and $A$ is the design matrix with $(A)_{ij}=(t_i)^j$ and $t$ the time argument. As is well known, the solution is $\alpha^*=(A^TA)^{-1}A^Tb$.

It is possible to use more features by adjusting $n_{expansion}$ to $3$. Due to visualization purposes this is the maximum number of features allowed in this example. Each timeseries $x_k$ is subsequently represented by a two dimensional feature vector

 $$feature(x_k)=(\alpha_1^k,\alpha_2^k).$$

Some exemplary features representing timeseries already colored in such a way as to indicate class membership are shown in the image below.

We embed these two-dimensional features into a higher dimensional space by replacing the feature $\alpha=(\alpha_1,\alpha_2)\in \mathbb{R}^2$ by $k(\alpha,\cdot)$ - an infinite sequence of powers in $\alpha_1$ and $\alpha_2$ indexed by a continuous argument. This is a quite broadly employable technique for handling nonlinearities and is usually called the 'kernel trick'.

```

# 4.1 Do a parametric regression to extract abscissa, slope, ... as the regression
# coefficients

# Preparation of design matrix
A = np.zeros((n_timesteps, n_expansion))
eye_mat = np.eye(n_expansion)
for k in range(n_expansion):
    Legendre_k = np.polynomial.legendre.Legendre(eye_mat[:,k])
    A[:,k]=Legendre_k(2*time-time[-1])

# Estimation via adjustment
coeffs_estimated = np.zeros((n_expansion, n_episodes))
x_estimated=np.zeros((n_timesteps, n_episodes))

for k in range(n_episodes):
    coeffs_estimated[:,k]=np.dot(np.dot(np.linalg.pinv(np.dot(np.transpose(A),A)),np.transpose(A)),x_measured[:,k])
    x_estimated[:,k]=A@coeffs_estimated[:,k]

# 4.2 Preprocessing to simplify interpretation
coeffs_estimated = coeffs_estimated.T  # Transpose such that X in R^Nxk
gt_labels = np.ones((coeffs_estimated.shape[0], 1))    # Prepare the labels to be y in {-1,1}
gt_labels[np.where(labels_true != 0), 0] = -1

print('Abscissa and slope have been extracted from timeseries via Least Squares and are \
assigned as features representing this timeseries.')

```

    Abscissa and slope have been extracted from timeseries via Least Squares and are assigned as features representing this timeseries.

# 5. Training the SVM -------------------------------------------------------

The simplest possible method to train a classifier would be to find the plane separating the features associated to the different classes. Typically this is not possible and one instead tries to find the plane minimizing the amount of misclassified examples and having the biggest distance to each of them. This is illustrated in the image below.

As indicated in section 4., the support vector machine goes one step further and embeds the 2-dimensional features into an infinite-dimensional Hilbert space $\mathcal{H}$ characterized by a kernel $k(\cdot,\cdot)$. The appropriate optimization formulation for finding the maximally discriminating hyperplane in that space is given by the problem of finding the optimal coefficients $\theta^*$ with

$$\begin{align} \theta^* =&\underset{\theta \in \mathbb{R}^{n_{episodes}}}{\operatorname{argmin}}\frac{1}{2} \sum_{i=1}^{n_{episodes}}\sum_{j=1}^{n_{episodes}}y_iy_jk(\alpha_i,\alpha_j)\theta_i\theta_j - \sum_{i=1}^{n_{episodes}}\theta_i \\ &s.t. ~~~~~~~~~0\le \theta_i \le c ~~~\forall i\in \{1, ... ,n_{episodes}\}\\& ~~~~~~~~~~~~~~~~\sum_{i=1}^{n_{episodes}}\theta_iy_i=0\end{align}$$

where $c$ is a constant specifying the amount of allowed irregularity in the decision boundary. This is a quadratic program that can be solved for the parameters $\theta$ using numerical python packages such as CVX, a convex optimization package developed and maintained by Stanford university (see references in slides). The final decision rule is then given by
$$\hat{y}=\operatorname{sign}\left(\sum_{j=1}^{n_{episodes}}\theta_jy_jk(\alpha_j,\alpha_j) + \frac{1}{n_{episodes}}\sum_{i=1}^{n_{episodes}}\left(y_i - \sum_{j=1}^{n_{episodes}}y_j\theta_jk(\alpha_i,\alpha_j)\right)\right).$$

More details can be found in Schoelkopf and Smola (1999), pp.203-209.

```

# 5.1. Prepare training step

# Split data into training and test set and initialize Kernel matrix
X_train, X_test, y_train, y_test = train_test_split(coeffs_estimated, gt_labels, test_size=0.2)
n_samples, n_features = X_train.shape
kernel_matrix = initialize_gram_matrix(X_train, 0.3)

# 5.2 execute training step by solving the following optimization problem using
# cvxopt convex optimization library.
# min 1/2 alpha^T Q alpha + e^T alpha
# s.t.
#  0 <= alpha <= c
#  A * alpha = 0

# Set optimization parameters
c = 1.0
lagr_threshold = 1e-7

# Define matrices in solver-readable form
Q = cvxopt.matrix(np.outer(y_train, y_train) * kernel_matrix)
e = cvxopt.matrix(np.ones(n_samples) * -1)
A = cvxopt.matrix(y_train, (1, n_samples))
b = cvxopt.matrix(0, tc='d')

# Assemble matrices and vectors to represent the inequalitie -a_i <= 0 and  a_i <= c.
G_std = cvxopt.matrix(np.identity(n_samples) * -1)
h_std = cvxopt.matrix(np.zeros(n_samples))
G_slack = cvxopt.matrix(np.identity(n_samples))
h_slack = cvxopt.matrix(np.ones(n_samples) * c)

# Assemble the matrices G_std and G_slack as well as the vectors h_std and h_slack
# into G and h such that the constraint on alpha is written as G alpha<= h.
G = cvxopt.matrix(np.vstack((G_std, G_slack)))
h = cvxopt.matrix(np.vstack((h_std, h_slack)))

# Call cvx convex optimization solver to solve the quadratic program
solution = cvxopt.solvers.qp(Q, e, G, h, A, b)

# 5.3 Assemble the SVM from the solution of the quadratic program. To do this,
# extract the support vectors and calculate the intercept.

# Extract support vectors by finding the vectors associated to the highest Langrange
# multipliers.
lagr_mult = np.ravel(solution['x'])
idx = lagr_mult > lagr_threshold
lagr_multipliers = lagr_mult[idx]

# Extract the support vector and get the corresponding labels
support_vectors = X_train[idx]
support_vector_labels = y_train[idx]

# Assemble the SVM and use it to predict the test data. Evaluate accuracy.
intercept = calculate_intercept(lagr_multipliers,support_vector_labels,support_vectors)
y_predicted = predict(X_test, lagr_multipliers, support_vector_labels, support_vectors, intercept)
accuracy = accuracy_score(y_test, y_predicted)
print("Accuracy on test data:", accuracy)

print('Support vector machine was successfully trained by solving a quadratic program for \
soft margin classification.')

```

         pcost       dcost       gap    pres   dres
     0: -1.1465e+01 -1.3406e+02  1e+02  9e-16  5e-16
     1: -1.2464e+01 -2.5006e+01  1e+01  1e-15  4e-16
     2: -1.3360e+01 -1.5135e+01  2e+00  3e-16  2e-16
     3: -1.3597e+01 -1.3886e+01  3e-01  1e-15  2e-16
     4: -1.3647e+01 -1.3704e+01  6e-02  3e-16  2e-16
     5: -1.3659e+01 -1.3668e+01  9e-03  2e-16  3e-16
     6: -1.3661e+01 -1.3663e+01  2e-03  2e-16  2e-16
     7: -1.3661e+01 -1.3661e+01  2e-04  8e-16  2e-16
     8: -1.3661e+01 -1.3661e+01  2e-05  6e-16  2e-16
     9: -1.3661e+01 -1.3661e+01  2e-07  3e-16  2e-16
    Optimal solution found.
    Accuracy on test data: 0.88
    Support vector machine was successfully trained by solving a quadratic program for soft margin classification.

# 6. Plots and illustrations ------------------------------------------------

Illustrate the results of the processing by plotting in figures 1 and 2 some exemplary sets of positively or negatively classified timeseries. Figure 3 shows a scatterplot of the features $(\alpha_1,\alpha_2)$ and figure 4 finally shows the decision boundary that is used by the trained SVM to predict whether the timeseries is associated to a stable or instable object.

If you are not satisfied with the decision rule and would rather detect a two-sided deformation, you can easily change this in section 1 by adjusting the anonymous function decision_rule.

```

# 6.1 Figure showing examples of timeseries that belong to an object that moeves (into the positive direction)
# and timeseries that belong to an object not moving (into the positive direction)

# Get indices of positive and negative examples
index_neg=np.where(labels_true==0)[0]
index_pos=np.where(labels_true==1)[0]

# Get the corresponding timeseries
x_measured_pos=x_measured[:,index_pos]
x_true_pos=x_true[:,index_pos]
x_measured_neg=x_measured[:,index_neg]
x_true_neg=x_true[:,index_neg]

# Plot some negative and some positive examples
plt.figure(1)
plt.plot(time,x_true_pos[:,range(10)],color='xkcd:crimson',linewidth=2)
for k in range(10):
    plt.scatter(time,x_measured_pos[:,k],s=60, facecolors='none', edgecolors='k')
plt.legend(['x_true','x_measured'])
plt.xlabel('Time')
plt.ylabel('Function value')
plt.title('Example: timeseries (moving in positive direction)')

plt.figure(2)
plt.plot(time,x_true_neg[:,range(10)],color='xkcd:cerulean',linewidth=2)
for k in range(10):
    plt.scatter(time,x_measured_neg[:,k],s=60, facecolors='none', edgecolors='k')
plt.legend(['x_true','x_measured'])
plt.xlabel('Time')
plt.ylabel('Function value')
plt.title('Example: timeseries (not moving in positive direction)')

# 6.2 Figure showing the features associated to the timeseries
plt.figure(3)
cm_bright = ListedColormap(['#FF0000', '#0000FF'])
plt.scatter(coeffs_estimated[:,0].reshape(-1), coeffs_estimated[:,1].reshape(-1), c=gt_labels.reshape(-1), cmap=cm_bright, s=30, edgecolors='k')
plt.xlabel('Regression coefficient 1')
plt.ylabel('Regression coefficient 2')
plt.title('Features and classes')

# 6.3 Figure showing the decision boundary distinguishing failure from non-failure cases

# Plot differently depending on dimensionality of the features
if coeffs_estimated.shape[1] == 2:

    # Create meshgrid on which to evaluate the predictions
    xx_1, xx_2 = make_meshgrid(coeffs_estimated[:,0], coeffs_estimated[:,1],h=0.1)
    X_plot = np.c_[xx_1.ravel(), xx_2.ravel()]
    y_plot = predict(X_plot, lagr_multipliers, support_vector_labels, support_vectors, intercept)
    y_plot = y_plot.reshape(xx_1.shape)

    # Plot the decision boundary.
    plt.figure()
    plt.contourf(xx_1, xx_2, y_plot, cmap=plt.cm.RdBu,alpha=0.8)
    plt.scatter(coeffs_estimated[:,0].reshape(-1), coeffs_estimated[:,1].reshape(-1), c=gt_labels.reshape(-1), cmap=cm_bright, s=30, edgecolors='k')
    plt.xlabel('Regression coefficient 1')
    plt.ylabel('Regression coefficient 2')
    plt.title('Decision boundary')

elif coeffs_estimated.shape[1] == 3:

    # Create meshgrid on which to evaluate the predictions
    xx_1, xx_2,xx_3 = make_meshgrid3(coeffs_estimated[:, 0], coeffs_estimated[:, 1],coeffs_estimated[:, 2], h=0.3)

    X_plot = np.c_[xx_1.ravel(), xx_2.ravel(), xx_3.ravel()]
    y_plot = predict(X_plot, lagr_multipliers, support_vector_labels, support_vectors, intercept)
    y_plot = y_plot.reshape(xx_1.shape)

    # Create a figure with axes for 3D plotting
    fig = plt.figure()
    ax = fig.gca(projection='3d')
    fig.suptitle("Classes and features in 3D")

    # Plot the different input points using 3D scatter plotting
    b1 = ax.scatter(coeffs_estimated[:, 0].reshape(-1), coeffs_estimated[:, 1].reshape(-1), coeffs_estimated[:, 2].reshape(-1), c=gt_labels.reshape(-1),
                cmap=plt.cm.coolwarm, s=30, edgecolors='k')

```


# Properties of Digit Sums

This notebook is known to be compatible with Julia 0.5. A basic foreknowledge of induction and mathematical proof will enrich your experience.

How would you check for divisibility by $3$ or $9$? You might know the trick. A number is divisible by $3$ if and only if the sum of its digits is divisible by $3$, and a number is divisible by $9$ if and only if the sum of its digits is divisible by $9$.

Why does this trick work? In this notebook, we'll look at some computational evidence, and then talk a bit about modular arithmetic, a computational strategy that we can use to prove these results.

## What is a digit sum?

Hold on. What do we mean by digit sum? Let's define the concept mathematically. Let $n$ be a positive whole number with base-$10$ representation $d_id_{i-1}d_{i-2}\dots d_3d_2d_1$, where all the $d_1$ are digits from $0$ to $9$, and we take the convention that $d_i\ne 0$. Then define the _sum of digits_ of $n$, denoted by $\operatorname{s}(n)$, as follows:

\begin{equation}
\operatorname{s}(n) = \sum_{k=1}^i d_k
\end{equation}

Why do we define the digits in reverse? That will make expressing $n$ as a sum of powers of $10$ easier. In fact, recall the definition of the base-$10$ system:

\begin{equation}
n = \sum_{k=1}^i d_k \cdot 10^{k-1}
\end{equation}

## How do I calculate it?

Let's first see how to extract the digits of a number, say, $1997$.

```julia
digits(1997)
```

    4-element Array{Int64,1}:
     7
     9
     9
     1

Wow, that's incredibly convenient. These digits are in the order we want too, counting from the right. We can compute each individual digit easily too:

```julia
d = digits(1997)
d[4]
```

    1

If you are used to a zero-based indexed programming language, note that Julia indexes arrays starting at $1$. So the fourth digit (counting from the right) is $1$. So far so good. Now how can we compute the sum of the digits?

```julia
sum(digits(1997))
```

    26

Doesn't get any easier than that! This syntax is so clean and descriptive that there is no point even making a helper function to compute it. Now that we have some syntax, let's verify the properties we alluded to initially. First, let's verify that a number is divisible by $9$ if and only if its sum of digits is divisible by $9$. To compute whether a number is divisible by $9$, we compute what it's remainder would be when divided by $9$, and check if it is zero. In Julia, the remainder operator is `%` (or the `rem` function), and the equality testing operator is `==`.

```julia
1997 % 9 == 0
```

    false

```julia
819 % 9 == 0
```

    true

Okay, so what numbers between $1$ and $1000$ are divisible by $9$? We can compute a vector of those numbers easily by using a `filter`. We want to find all elements $n$ in `1:1000` where `n % 9 == 0`. Here's how to write that in code:

```julia
A = filter(n -> n % 9 == 0, 1:1000)
```

    111-element Array{Int64,1}:
       9
      18
      27
      36
      45
      54
      63
      72
      81
      90
      99
     108
     117
       ⋮
     900
     909
     918
     927
     936
     945
     954
     963
     972
     981
     990
     999

Now let's do the same computation, using the sum of digits as the divisibility test:

```julia
B = filter(n -> sum(digits(n)) % 9 == 0, 1:1000)
```

    111-element Array{Int64,1}:
       9
      18
      27
      36
      45
      54
      63
      72
      81
      90
      99
     108
     117
       ⋮
     900
     909
     918
     927
     936
     945
     954
     963
     972
     981
     990
     999

```julia
A == B
```

    true

Success. We've computed the multiples of $9$ two ways, and both times we got the same result. But that's not a proof! **rest to come**


# Lorenz System of Differential Equations

In 1963, Edward Lorenz developed a mathematical model consisting of three, non-linear, ordinary differential equations to study atmospheric convection. Though the set of differential equations are *deterministic*, the Lorenz system is perhaps most noteworthy for exhibiting chaotic behavior. In other words, small perturbations in the initial conditions result in large variations in the dynamics of the system, what is often referred to in lay terms as the 'butterfly effect'.

The Lorenz system of differential equations are

\begin{align}
\dot{x} & = \sigma(y-x) \\
\dot{y} & = \rho x - y - xz \\
\dot{z} & = -\beta z + xy
\end{align}

Commonly used as an introduction to the interactive data visualization in Jupyter Notebooks, the following code is described in further detail [here](https://ipywidgets.readthedocs.io/en/stable/examples/Lorenz%20Differential%20Equations.html), [here](https://jakevdp.github.io/blog/2013/02/16/animating-the-lorentz-system-in-3d/), and [here](https://matplotlib.org/3.1.0/gallery/mplot3d/lorenz_attractor.html). For more information about the Lorenz system, refer to [Wikipedia](https://en.wikipedia.org/wiki/Lorenz_system).

Our interest in the Lorenz system is merely to state that chaotic behavior is an attribute of many non-linear, differential systems of equations. It is important to note that describing a system of equations as chaotic does not mean seeking solutions is intractable, it only means we must rely on statistics to describe events in  a more meaningful way. I mention this because as students of fluid mechanics, we may mistakenly think that modeling the turbulent nature of flows to be something *outside* of the ability of the Navier-Stokes equations. And yes, while finding **deterministic** solutions to Navier-Stokes equations is often *not* feasible (computationally), they certainly are *possible*.

```python
%matplotlib inline

from ipywidgets import interact, interactive
from IPython.display import clear_output, display, HTML

import numpy as np
from scipy import integrate

from matplotlib import pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
from matplotlib.colors import cnames
from matplotlib import animation
```

```python
def solve_lorenz(N=10, angle=0.0, max_time=4.0, sigma=10.0, beta=8./3, rho=28.0):
    """
    Solve the Lorenz system of equations and plot the trajectories.

    Args:
        N (int) : Number of trajectories to simulate. Starting points are
            randomly selected from a uniform distribution [-15,15].
        angle (float) : View angle for visualization.
        max_time (float) :
        sigma (float) :  Lorenz parameter proportional to the Prandtl number.
        beta (float) : Lorenz parameter proportional to the Rayleigh number.
        rho (float) : Lorenz parameter proportional to the scale of the convective layer.
    """
    fig = plt.figure()
    ax = fig.add_axes([0, 0, 1, 1], projection='3d')
    ax.axis('off')

    # prepare the axes limits
    ax.set_xlim((-25, 25))
    ax.set_ylim((-35, 35))
    ax.set_zlim((5, 55))

    def lorenz_deriv(x_y_z, t0, sigma=sigma, beta=beta, rho=rho):
        """Compute the time-derivative of a Lorenz system."""
        x, y, z = x_y_z
        return [sigma * (y - x), x * (rho - z) - y, x * y - beta * z]

    # Choose random starting points, uniformly distributed from -15 to 15
    np.random.seed(1)
    x0 = -15 + 30 * np.random.random((N, 3))

    # Solve for the trajectories
    t = np.linspace(0, max_time, int(250*max_time))
    x_t = np.asarray([integrate.odeint(lorenz_deriv, x0i, t)
                      for x0i in x0])

    # choose a different color for each trajectory
    colors = plt.cm.jet(np.linspace(0, 1, N))

    for i in range(N):
        x, y, z = x_t[i,:,:].T
        lines = ax.plot(x, y, z, '-', c=colors[i])
        plt.setp(lines, linewidth=2)

    ax.view_init(30, angle)
    plt.show()

    return t, x_t
```

```python
t, x_t = solve_lorenz(angle=0, N=2)
```

Near $\rho = 28$, $\sigma = 10$ and $\beta = 8/3$, the system exhibits chaotic behavior. Trajectories with values of $\rho$, $\sigma$, and $\beta$ near these will evolve significantly different with only small perturbations $(\approx 10^{-5})$ in initial position. Initially, the two trajectories will be coincident, but in time, they will diverge.

Using the interactive widget, you should be able to observe this. For values of $\rho < 1$, the solutions are stable.

```python
w = interactive(solve_lorenz, angle=(0.,360.), N=(0,50), sigma=(0.0,50.0), rho=(0.0,50.0))
display(w)
```

    interactive(children=(IntSlider(value=10, description='N', max=50), FloatSlider(value=0.0, description='angle'…

```python
t, x_t = w.result
```

```python
xyz_avg = x_t.mean(axis=1)
```

```python
import seaborn as sns
sns.set()

fig, (ax1, ax2) = plt.subplots(nrows=1, ncols=2, figsize=(11, 6))

ax1.hist(xyz_avg[:,0])
ax1.set_title('Average $x(t)$', fontsize=15)

ax2.hist(xyz_avg[:,1])
ax2.set_title('Average $y(t)$', fontsize=15)
```


Overview of "A Quantum Approximate Optimization Algorithm" written by Edward Farhi, Jeffrey Goldstone and Sam Gutmann.

# Introduction:

Combinatorial optimization problems attempt to optimize an objective function over *n* bits with respect to *m* clauses.  The bits are grouped into a string $z = z_1z_2...z_n$, while clauses are constraints on a subset of the bits, satisfied for some strings, not satisfied for others.  The objective function, then, is defined as

\begin{equation} \tag{1}
C(z) = \sum_{\alpha=1}^{m} C_{\alpha}(z)
\end{equation}

where $C_{\alpha}(z) = 1$ if *z* satisfies clause $\alpha$, and is 0 otherwise.  Note that $C_{\alpha}(z)$ typically will only depend on a few of the bits in the string.

Now, the goal of approximate optimization is to find a string *z* for which $C(z)$ is close to the maximum value $C$ takes over all strings.  This paper presents a quantum algorithm, paired with classical pre-processing, for approximate optimization.

A quantum computer with *n* qubits works in a $2^n$ dimensional Hilbert space, with basis vectors denoted by $|z>$ where $z=1,2,3,...2^n$, known as the computational basis.  In this framework, we view $(1)$ as an operator, which is diagonal in the computational basis.

Next, a few more operators must be defined in order to map the approximate optimization problem onto the quantum comptuer.  First, we define a unitary operator $U(C, \gamma)$, which depends on the objective function $C$ and an angle $\gamma$

\begin{equation} \tag{2}
U(C,\gamma) = e^{-i{\gamma}C} = \prod_{\alpha=1}^{m} e^{-i{\gamma}C_\alpha}
\end{equation}

The second equality in $(2)$ is permitted since all $C_\alpha$'s commute with another (i.e. they are all diagonal in the same basis).  Note that since $C$ has integer eigenvalues we can restrict the values of $\gamma$ to lie between $0$ and $2\pi$.
Next, we define an operator $B$, which is the sum of all single bit-flip operators, represented by the Pauli-x matrix

\begin{equation} \tag{3}
B = \sum_{j=1}^{n} \sigma^{x}_j
\end{equation}

Then we define another angle-dependent unitary operator as a product of commuting one-bit operators

\begin{equation} \tag{4}
U(B,\beta) = e^{-i{\beta}B} = \prod_{j=1}^{n} e^{-i{\beta}\sigma^{x}_j}
\end{equation}

where $\beta$ takes values between $0$ and $\pi$.
The initial state of the qubits will be set to a uniform superposition over all computational basis states, defined as:

\begin{equation} \tag{5}
|s> = \frac{1}{2^n} \sum_{z} |z>
\end{equation}

Next, we define an integer $p \ge 1$ which will set the quality of the approximate optimization; the higher $p$ the better approximation can be attained.  For a given $p$, we define a total of $2p$ angles $\gamma_1 . . . \gamma_p \equiv \pmb\gamma$ and $\beta_1 . . . \beta_p \equiv \pmb\beta$ which will define an angle-dependent quantum state for the qubits

\begin{equation} \tag{6}
|\pmb\gamma, \pmb\beta> = U(B,{\beta}_p)U(C,{\gamma}_p)...U(B,{\beta}_1)U(C,{\gamma}_1)|s>
\end{equation}

Notice that there are $p$ $C$-dependent unitaries, each of which require $m$ rotation gates on local sets of qubits.  In the worst case, each $C$-dependent unitary must be run in a different "moment" (when referring to a quantum circuit, a moment is essentially one "clock cycle" in which a group of operators acting on various qubits can be performed simulatneously), and thus the $C$-unitaries can be implemented with a circuit depth of $\mathcal{O}(mp)$.  Meanwhile, there are $p$ $B$-dependent unitaries, each of which only involve single qubit-operators and thus can all be applied in one "moment".  Therefore, the $B$-unitaries have a circuit depth of $\mathcal{O}(mp)$.  This means that the final state we seek can be prepared on the quantum computer with a circuit depth of $\mathcal{O}(mp + p)$.

Next, we define $F_p$ as the expectation value of $C$ in this state:
\begin{equation} \tag{7}
F_p(\pmb\gamma, \pmb\beta) = <\pmb\gamma, \pmb\beta|C|\pmb\gamma, \pmb\beta>
\end{equation}

and define the maximum value that $F_p$ takes over all angles as
\begin{equation} \tag{8}
M_p = max_{\gamma,\beta}F_p(\pmb\gamma, \pmb\beta)
\end{equation}

Finally, with all these terms defined, we can lay out an algorithm for approximate optimization.
1. Pick integer $p$ and determine a set of 2$p$ angles $\{ \pmb\gamma, \pmb\beta \}$ which maximize $F_p$.
2. Prepare the state $|\gamma, \beta>$ on the quantum computer.
3. Measure the qubits in the computational basis to obtain the string $z$ and evaluate $C(z)$
4.  Perform step 3 repeatedly with the same angles to obtain a string $z$ such that $C(z)$ is very near or greater than $F_p(\pmb\gamma, \beta)$

The main roadblock to this algorithm is to find the optimal set of angles $\{ \pmb\gamma, \pmb\beta \}$.  One method, if $p$ does not grow with $n$, is to use brute force by running the quantum computer with values of $\{ \pmb\gamma, \pmb\beta \}$ chosen on a fine grid on the compact set $[0,2\pi]^p \times [0,\pi]^p$ in order to find the values of angles that produce the maximum $F_p$.  The paper also presents a method using classical pre-processing to determine the optimal angles.  However, as this overview is intended to focus on the quantum computational part of the algorithm, we will assume that optimal angles $\{ \pmb\gamma, \pmb\beta \}$ have been determined in *some* fashion, and illustrate in detail how the quantum computing part of this algorithm works.

# Example Problem: MaxCut for Graphs with Bounded Degree

We will now examine the quantum part of the quantum approximation optimization algorithm (QAOA) using the MaxCut problem for graphs with a bounded degree as an example optimization problem.  The input is a graph with $n$ vertices and an edge set $\{ \langle jk \rangle \}$ of size $m$.  The $n$ vertices are mapped to $n$ qubits, while the $m$ edges of the edge set represent the $m$ clauses of the combinatorial optimization problem.  The goal is to maximize the objective function defined as

\begin{equation} \tag{9}
C = \sum_{\langle jk \rangle } C_{\langle jk \rangle }
\end{equation}

where

\begin{equation} \tag{9}
C_{\langle jk \rangle } = \frac{1}{2}(-\sigma^z_j \sigma^z_k + 1)
\end{equation}

Each clause $C_{\langle jk \rangle }$ is thus equal to 1 when qubits $j$ and $k$ have spins pointing in opposite directions along the *z*-direction, and equal to 0 when the spins are in the same direction.

## MaxCut for Bounded Graph of Degree 2 with 4 Vertices
Below, we give the Cirq implementation of a QAOA for the MaxCut problem for a regular graph of degree 2 with $n=4$ vertices and $m=4$ edges given as $\langle 1,2 \rangle ,\langle 2,3 \rangle ,\langle 3,4 \rangle ,\langle 4,1 \rangle $.  Note that this graph is just a ring. For simplicity, we choose $p=1$ and arbitrarily choose values of $\beta_1$ and $\gamma_1$ (though in a real implementation the optimal angles must be found either by brute force or clever classical pre-processing).

To run, press the play button in the upper left-hand corner of the code boxes below.  The code boxes must be run in sequential order, starting with the code box that imports Cirq and other necessary Python libraries.

```python
!pip install git+https://github.com/quantumlib/Cirq
import cirq
import numpy as np
import matplotlib.pyplot as plt
import cmath
import scipy.linalg
```

    Collecting git+https://github.com/quantumlib/Cirq
      Cloning https://github.com/quantumlib/Cirq to /tmp/pip-req-build-1qjwfhm9
      Running command git clone -q https://github.com/quantumlib/Cirq /tmp/pip-req-build-1qjwfhm9
    Requirement already satisfied (use --upgrade to upgrade): cirq==0.9.0.dev0 from git+https://github.com/quantumlib/Cirq in /usr/local/lib/python3.6/dist-packages
    Requirement already satisfied: freezegun~=0.3.15 in /usr/local/lib/python3.6/dist-packages (from cirq==0.9.0.dev0) (0.3.15)
    Requirement already satisfied: google-api-core[grpc]<2.0.0dev,>=1.14.0 in /usr/local/lib/python3.6/dist-packages (from cirq==0.9.0.dev0) (1.16.0)
    Requirement already satisfied: matplotlib~=3.0 in /usr/local/lib/python3.6/dist-packages (from cirq==0.9.0.dev0) (3.2.1)
    Requirement already satisfied: networkx~=2.4 in /usr/local/lib/python3.6/dist-packages (from cirq==0.9.0.dev0) (2.4)
    Requirement already satisfied: numpy~=1.16 in /usr/local/lib/python3.6/dist-packages (from cirq==0.9.0.dev0) (1.18.4)
    Requirement already satisfied: pandas in /usr/local/lib/python3.6/dist-packages (from cirq==0.9.0.dev0) (1.0.3)
    Requirement already satisfied: protobuf~=3.11.0 in /usr/local/lib/python3.6/dist-packages (from cirq==0.9.0.dev0) (3.11.3)
    Requirement already satisfied: requests~=2.18 in /usr/local/lib/python3.6/dist-packages (from cirq==0.9.0.dev0) (2.23.0)
    Requirement already satisfied: sortedcontainers~=2.0 in /usr/local/lib/python3.6/dist-packages (from cirq==0.9.0.dev0) (2.1.0)
    Requirement already satisfied: scipy in /usr/local/lib/python3.6/dist-packages (from cirq==0.9.0.dev0) (1.4.1)
    Requirement already satisfied: sympy in /usr/local/lib/python3.6/dist-packages (from cirq==0.9.0.dev0) (1.1.1)
    Requirement already satisfied: typing_extensions in /usr/local/lib/python3.6/dist-packages (from cirq==0.9.0.dev0) (3.6.6)
    Requirement already satisfied: dataclasses in /usr/local/lib/python3.6/dist-packages (from cirq==0.9.0.dev0) (0.7)
    Requirement already satisfied: six in /usr/local/lib/python3.6/dist-packages (from freezegun~=0.3.15->cirq==0.9.0.dev0) (1.12.0)
    Requirement already satisfied: python-dateutil!=2.0,>=1.0 in /usr/local/lib/python3.6/dist-packages (from freezegun~=0.3.15->cirq==0.9.0.dev0) (2.8.1)
    Requirement already satisfied: setuptools>=34.0.0 in /usr/local/lib/python3.6/dist-packages (from google-api-core[grpc]<2.0.0dev,>=1.14.0->cirq==0.9.0.dev0) (46.1.3)
    Requirement already satisfied: google-auth<2.0dev,>=0.4.0 in /usr/local/lib/python3.6/dist-packages (from google-api-core[grpc]<2.0.0dev,>=1.14.0->cirq==0.9.0.dev0) (1.7.2)
    Requirement already satisfied: pytz in /usr/local/lib/python3.6/dist-packages (from google-api-core[grpc]<2.0.0dev,>=1.14.0->cirq==0.9.0.dev0) (2018.9)
    Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in /usr/local/lib/python3.6/dist-packages (from google-api-core[grpc]<2.0.0dev,>=1.14.0->cirq==0.9.0.dev0) (1.51.0)
    Requirement already satisfied: grpcio<2.0dev,>=1.8.2; extra == "grpc" in /usr/local/lib/python3.6/dist-packages (from google-api-core[grpc]<2.0.0dev,>=1.14.0->cirq==0.9.0.dev0) (1.28.1)
    Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib~=3.0->cirq==0.9.0.dev0) (2.4.7)
    Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.6/dist-packages (from matplotlib~=3.0->cirq==0.9.0.dev0) (0.10.0)
    Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib~=3.0->cirq==0.9.0.dev0) (1.2.0)
    Requirement already satisfied: decorator>=4.3.0 in /usr/local/lib/python3.6/dist-packages (from networkx~=2.4->cirq==0.9.0.dev0) (4.4.2)
    Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from requests~=2.18->cirq==0.9.0.dev0) (3.0.4)
    Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.6/dist-packages (from requests~=2.18->cirq==0.9.0.dev0) (1.24.3)
    Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.6/dist-packages (from requests~=2.18->cirq==0.9.0.dev0) (2.9)
    Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.6/dist-packages (from requests~=2.18->cirq==0.9.0.dev0) (2020.4.5.1)
    Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.6/dist-packages (from sympy->cirq==0.9.0.dev0) (1.1.0)
    Requirement already satisfied: rsa<4.1,>=3.1.4 in /usr/local/lib/python3.6/dist-packages (from google-auth<2.0dev,>=0.4.0->google-api-core[grpc]<2.0.0dev,>=1.14.0->cirq==0.9.0.dev0) (4.0)
    Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.6/dist-packages (from google-auth<2.0dev,>=0.4.0->google-api-core[grpc]<2.0.0dev,>=1.14.0->cirq==0.9.0.dev0) (0.2.8)
    Requirement already satisfied: cachetools<3.2,>=2.0.0 in /usr/local/lib/python3.6/dist-packages (from google-auth<2.0dev,>=0.4.0->google-api-core[grpc]<2.0.0dev,>=1.14.0->cirq==0.9.0.dev0) (3.1.1)
    Requirement already satisfied: pyasn1>=0.1.3 in /usr/local/lib/python3.6/dist-packages (from rsa<4.1,>=3.1.4->google-auth<2.0dev,>=0.4.0->google-api-core[grpc]<2.0.0dev,>=1.14.0->cirq==0.9.0.dev0) (0.4.8)
    Building wheels for collected packages: cirq
      Building wheel for cirq (setup.py) ... done
      Created wheel for cirq: filename=cirq-0.9.0.dev0-cp36-none-any.whl size=1407599 sha256=32feab921e73df656a3323d61449bbf0671f5b88a9bc93ee4975c07b275d1737
      Stored in directory: /tmp/pip-ephem-wheel-cache-otlqifme/wheels/c9/f4/ee/029123a49c5e2d75d08c2a9f937e207b88f045901db04632a7
    Successfully built cirq

```python
#This code runs the QAOA for the example of MaxCut on a 2-regular graph with
#n=4 vertices and m=4 clauses
#The graph is a ring

# define the length of grid.
length = 2
nqubits = length**2
# define qubits on the grid.
#in this case we have 4 qubits on a 2x2 grid
qubits = [cirq.GridQubit(i, j) for i in range(length) for j in range(length)]
#instantiate a circuit
circuit = cirq.Circuit()

#apply Hadamard gate to all qubits to define the initial state
#here, the initial state is a uniform superposition over all basis states
circuit.append(cirq.H(q) for q in qubits)

#here, we use p=1
#define optimal angles beta and gamma computed by brute force
#or through classical pre-processing
#beta in [0,pi]
#gamma in [0,2*pi]
beta = 0.2
gamma = 0.4

#define operators for creating state |gamma,beta> = U_B*U_C*|s>
#define U_C operator = Product_<jk> { e^(-i*gamma*C_jk)} and append to main circuit
coeff = -0.5 #coefficient in front of sigma_z operators in the C_jk operator
U_C = cirq.Circuit()
for i in range(0,nqubits):
  U_C.append(cirq.CNOT.on(qubits[i],qubits[(i+1)%nqubits]))
  U_C.append(cirq.rz(2.0*coeff*gamma).on(qubits[(i+1)%nqubits]))
  U_C.append(cirq.CNOT.on(qubits[i],qubits[(i+1)%nqubits]))
circuit.append(U_C)

#define U_B operator = Product_j {e^(-i*beta*X_j)} and append to main circuit
U_B = cirq.Circuit()
for i in range(0,nqubits):
  U_B.append(cirq.H(qubits[i]))
  U_B.append(cirq.rz(2.0*beta).on(qubits[i]))
  U_B.append(cirq.H(qubits[i]))
circuit.append(U_B)

#add measurement operators for each qubit
#measure in the computational basis to get the string z
for i in range(0,nqubits):
  circuit.append(cirq.measure(qubits[i],key=str(i)))

#run circuit in simulator to get the state |beta,gamma> and measure in the
#computational basis to get the string z and evaluate C(z)
#repeat for 100 runs and save the best z and C(z)
#simulator = cirq.google.XmonSimulator()
simulator = cirq.Simulator()
reps = 100
results = simulator.run(circuit, repetitions=reps)

#get bit string z from results
best_z = [None]*nqubits
best_Cz = 0.0
all_Czs = []
for i in range(0,reps):
  z = []
  for j in range (0,nqubits):
    if (results.measurements[str(j)][i]):
      z.append(1)
    else:
      z.append(-1)

  #compute C(z)
  Cz = 0.0
  for j in range(0,nqubits):
    Cz += 0.5*(-1.0*z[j]*z[(j+1)%nqubits] + 1.0)
  all_Czs.append(Cz)

  #store best values for z and C(z)
  if (Cz > best_Cz):
    best_Cz = Cz
    best_z = z

#print best string z and corresponding C(z)
print("Best z")
print(best_z)
print("Best objective")
print(best_Cz)

plt.plot(all_Czs)
plt.xlabel("Run")
plt.ylabel("C(z) value")
plt.show()

#print a diagram of the circuit
print(circuit)
```

## Explanation of Code

In what follows, we give a detailed description of the above code.
###Define the qubits
First, the number of qubits ($n=4$) is defined on a 2x2 grid and we instantiate the circuit object, which will contain the quantum program we wish to run on the qubits.
### Prepare the Initial State
Next, we prepare the initial state of the system, a uniform superposition over all basis states.  Thus, we append a Hadamard gate acting on each qubit to the circuit object.  As stated above, $p$ is set to 1 and $\beta$ and $\gamma$ are arbitrarily chosen in this implementation, however, in a real run of the algorithm, one would need to find the optimal values of these angles through brute force or classical pre-processing.
Next, recall that the goal is to create the state

\begin{equation} \tag{10}
|\gamma, \beta \rangle = U(B,\beta)U(C,\gamma)|s \rangle
\end{equation}

Thus, we must define the unitary operators $U(C,\gamma)$ and $U(B,\beta)$ which will be appended to our circuit object, after the state initialization gates (Hadamard gates).

###Build the operator $U(C, \gamma)$
We begin by building up the operator $U(C, \gamma)$, which is defined as:

\begin{equation} \tag{11}
U(C,\gamma) = e^{-i{\gamma}C} = \prod_{ \langle jk \rangle} e^{-i{\gamma}C_{<jk>}} = \prod_{\langle jk \rangle} e^{-i\frac{\gamma}{2}(-\sigma^z_j\sigma^z_k +1)}
\end{equation}

Examaning the right-hand side of $(11)$, note that we can simplify this by dropping the identity term (the second term in the parantheses in the exponent), as this simple phase term will drop out once we make measurements of the qubits.  Intuitively, an identity operator should have no effect on the system.  Thus, we construct $U(C,\gamma)$ as the following product of exponentials:

\begin{equation} \tag{11}
U(C,\gamma) = e^{-i\frac{\gamma}{2}(-\sigma^z_1\sigma^z_2)}e^{-i\frac{\gamma}{2}(-\sigma^z_2\sigma^z_3)}e^{-i\frac{\gamma}{2}(-\sigma^z_3\sigma^z_4)}e^{-i\frac{\gamma}{2}(-\sigma^z_4\sigma^z_1)}
\end{equation}

Each exponential acts on one pair of qubits, and can be translated into a quantum circuit using $Rz(\theta)$ gates, which perform rotations of the spin about the *z*-axis through an angle of $\theta$, and $CNOT$ gates which essentially entangle the two qubits in the pair.

 As a specific example we examine how the first expoential operator in $(11)$ gets translated into quantum logic gates:

\begin{eqnarray*} \tag{12}
    e^{-i\frac{\gamma}{2}(-\sigma^z_1\sigma^z_2)}  & \rightarrow  &         CNOT[1,2]\\
    &  & I[1] \otimes Rz(2*-0.5*\gamma)[2]\\
    &  & CNOT[1,2]
\end{eqnarray*}

  Here the right side indicates three gates are used to perform this operator: the $CNOT$ gate acts on qubits 1 and 2, then an $Rz(\theta)$ gate is acted on qubit 2, and finally, another $CNOT$ gate is acted on qubits 1 and 2.  To show why this works, we simply need to show that the matrix representation of the left and right-hand sides of $(12)$ are the same.

We derive the matrix for the left-hand side first.  Since this operator acts on two qubits, it acts on a $2^2 = 4$ dimensional Hilbert space, and thus is represented by a 4x4 matrix.  The term in the exponential, $-i\frac{\gamma}{2}(-\sigma^z_1\sigma^z_2)$, is defined by the tensor product of the two Pauli-z terms multiplied by a coefficient:

\begin{equation} \tag{13}
  -i\frac{\gamma}{2}(-\sigma^z_1\sigma^z_2) = i\frac{\gamma}{2}\sigma^z_1 \otimes \sigma^z_2
\end{equation}

Now
\begin{equation} \tag{14}
  \sigma^z_i = \begin{bmatrix}
    1 & 0\\
    0 & -1
\end{bmatrix}
\end{equation}

So

\begin{equation} \tag{15}
  i\frac{\gamma}{2}\sigma^z_1 \otimes \sigma^z_2=
  i\frac{\gamma}{2}\begin{bmatrix}
    1 & 0 & 0 & 0\\
    0 & -1 & 0 & 0 \\
    0 & 0 & -1 & 0 \\
    0 & 0 & 0 & 1
\end{bmatrix}
\end{equation}

And thus

\begin{equation} \tag{16}
  e^{i\frac{\gamma}{2}\sigma^z_1 \otimes \sigma^z_2}=
  \begin{bmatrix}
    e^{i\frac{\gamma}{2}} & 0 & 0 & 0\\
    0 & e^{-i\frac{\gamma}{2}} & 0 & 0 \\
    0 & 0 & e^{-i\frac{\gamma}{2}} & 0 \\
    0 & 0 & 0 & e^{i\frac{\gamma}{2}}
\end{bmatrix}
\end{equation}

Now we derive the matrix representation of the right-hand side of (12).  First the $CNOT$ gate can be written as:

\begin{equation} \tag{17}
  CNOT=
  \begin{bmatrix}
    1 & 0 & 0 & 0\\
    0 & 1 & 0 & 0 \\
    0 & 0 & 0 & 1 \\
    0 & 0 & 1 & 0
\end{bmatrix}
\end{equation}

Next, the $Rz(\theta)$ term is defined as
\begin{equation} \tag{18}
  Rz(\theta)= e^{-iZ\frac{\theta}{2}} =
  \begin{bmatrix}
    e^{-i\frac{\theta}{2}} & 0 \\
    0 & e^{i\frac{\theta}{2}}
\end{bmatrix}
\end{equation}

Thus

\begin{eqnarray*}
  I \otimes Rz(2*-0.5*\gamma) & = &
  \begin{bmatrix}
    1 & 0 \\
    0 & 1
  \end{bmatrix} \otimes
  \begin{bmatrix}
    e^{-i\frac{\gamma}{2}} & 0 \\
    0 & e^{i\frac{\gamma}{2}}
  \end{bmatrix} \\
  & = &  \begin{bmatrix}  \tag{19}
    e^{i\frac{\gamma}{2}} & 0 & 0 & 0\\
    0 & e^{-i\frac{\gamma}{2}} & 0 & 0 \\
    0 & 0 & e^{i\frac{\gamma}{2}} & 0 \\
    0 & 0 & 0 & e^{-i\frac{\gamma}{2}}
\end{bmatrix}
\end{eqnarray*}

Putting together the three gates we get:

\begin{equation} \tag{20}
  \begin{bmatrix}
    1 & 0 & 0 & 0\\
    0 & 1 & 0 & 0 \\
    0 & 0 & 0 & 1 \\
    0 & 0 & 1 & 0
\end{bmatrix} * \begin{bmatrix}
    e^{i\frac{\gamma}{2}} & 0 & 0 & 0\\
    0 & e^{-i\frac{\gamma}{2}} & 0 & 0 \\
    0 & 0 & e^{i\frac{\gamma}{2}} & 0 \\
    0 & 0 & 0 & e^{-i\frac{\gamma}{2}}
\end{bmatrix} *   \begin{bmatrix}
    1 & 0 & 0 & 0\\
    0 & 1 & 0 & 0 \\
    0 & 0 & 0 & 1 \\
    0 & 0 & 1 & 0
\end{bmatrix} =   \begin{bmatrix}
    e^{i\frac{\gamma}{2}} & 0 & 0 & 0\\
    0 & e^{-i\frac{\gamma}{2}} & 0 & 0 \\
    0 & 0 & e^{-i\frac{\gamma}{2}} & 0 \\
    0 & 0 & 0 & e^{i\frac{\gamma}{2}}
\end{bmatrix}
\end{equation}

Notice the $(20)$ is identical to $(16)$, and thus we have proven that these three gates do indeed implement the first operator in $U(C,\gamma)$.  The other operators are built analogously except that they act on different pairs of qubits.  Once the sub-circuit for $U(C,\gamma)$ is defined, we append it to our main circuit.

###Build the operator $U(B, \beta)$

In this example $U(B,\beta)$ is given by
\begin{equation} \tag{21}
U(B,\beta) = e^{-i{\beta}\sigma^x_1}e^{-i{\beta}\sigma^x_2} e^{-i{\beta}\sigma^x_3} e^{-i{\beta}\sigma^x_4}
\end{equation}

Again we examine how to convert the first exponential operator into quantum logic gates:
\begin{eqnarray*} \tag{22}
    e^{-i{\beta}\sigma^x_1}  & \rightarrow  &  H[1]\\
    &  & Rz(2*\beta)[1]\\
    &  & H[1]
\end{eqnarray*}

The derivation of this is much simpler.  Note that $\sigma^x$ is diagonal in the x-basis, but not in the computational(z)-basis.  Since exponentiated operators are easy to carry out when the operator is diagonal, we apply a Hadamard gate on the qubit to rotate it into the x-basis, where this operator is now diagonal and represented by the $Rz(\theta)$ gate.  We apply said $Rz(\theta)$ gate to the qubit, and then apply a second Hadamard gate to the qubit to rotate it back into the computational basis.  Analogous gate sets are applied to all other qubits to carry out the full $U(B,\beta)$ operator.  Once the sub-circuit for $U(B,\beta)$ is defined, we append it to our main circuit.

###Measurement and Simulation
At this point, the main circuit now contains gates for initial state preparation and application of the $U(C,\gamma)$ and $U(B,\beta)$ unitary operators to get us into the final state
\begin{equation} \tag{23}
|\gamma, \beta \rangle = U(B,\beta)U(C,\gamma)|s \rangle
\end{equation}

The next step of the algorithm is to measure each of the qubits to get the string $z$ and then evaluate $C(z)$.  Measurement is performed by appending a measurement operator on the computational basis to each of the qubits in our main circuit.  A simulation is carried out by instantiating a Simulator.  We then input the main circuit to this simulator, and since we will want to run this circuit many times, we can give an optional argument for the number of repetitions (in this case, 10).

The circuit will then be simulated 10 times.  The paper says that $m log(m)$ repetitions should suffice, and since in this case $m=4$, 10 repetitions are more than enough. The results of all runs are stored in the variable "results".  We can then evaluate the value of $C(z)$ for each of the runs and keep a running tab on which string $z$ gives the maximum value of $C(z)$, each of which is printed at the end of the program.  A diagram of the circuit is also output at the end of the program to give the reader a clearer understanding of the program that is being run on the quantum computer.

To understand how this approximates the optimization of the MaxCut problem, note that $z = z_1z_2...z_n$ is a string, where each $z_i$ equals 0 or 1.  In the MaxCut problem, one wants to find a subset of the vertices $S$ such that the number of edges between $S$ and the complementary subset is maximal.  The final $z$ string we measure defines whether each vertex $i$ is in the subset $S$ (say, $z_i = 1$) or in the complement ($z_i = 0$).  $C(z)$ gives the number of edges that exist between the subset S and its complement.  So in our example, the 4-vertex ring, after enough repetitions, our QAOA finds the correct maximum cut of $C(z)=4$ and returns a string of alternating 0's and 1's, indicating the graph is maximally cut when alternating vertices are grouped into subsets.  This toy example was small and simple, which is why the QAOA was able to return the true optimal solution.  However, in more complex combinatorial optimization problems, this will not always be the case.  It may be necessary to increase the value of $p$, which in this simple case we took to just be 1.

## MaxCut for Bounded Graph of Degree 2 with 16 Vertices and Brute Force Angle Optimization

In the previous example with 4 vertices, the algorithm almost always returned the optimal cut, despite our random selection of values for $\beta$ and $\gamma$.  Clearly, the problem was so small that the search space of possible solutions was easily navigated over the repeated runs.  However, if we move to a larger problem with 16 vertices, it becomes more important to find optimal values for $\beta$ and $\gamma$ in order to find the optimal cut with high probability.

Below is some code that performs a grid search over values for the angles to find their optimal values.  Now, since $m=16$ it becomes necessary to increase the number of repetitions each circuit is run for to be $\mathcal{O}(16log(16))$.  We, therefore, increase the number of repetitions to 100 in an attempt to keep it low enough for the code below not to take more than 5 minutes to run.  Upon completion, that program prints the resulting optimal values for $\beta$, $\gamma$, $C(z)$, and $z$.  It also plots a graph of how the value of $C(z)$ changes throughout the grid search as various values of $\beta$ and $\gamma$ are swept through.  It is clear from the plot that there is indeed an important angle-dependence now that our problem has more vertices, and thus resides in a larger search space.

**Note: Code below may take up to 5 minutes to run!**

```python
#This code runs the QAOA, including optimization of the angles beta and gamma
#for the example of MaxCut on a 2-regular graph with n=16 vertices and m=16 clauses
#Brute force optimization of the angles beta ang gamma is performed by
#a grid search over their possible values

# define the length of grid.
length = 4
nqubits = length**2
# define qubits on the grid.
#in this case we have 16 qubits on a 4x4 grid
qubits = [cirq.GridQubit(i, j) for i in range(length) for j in range(length)]

#here, we use p=1
#search for optimal angles beta and gamma by brute force
#beta in [0,pi]
#gamma in [0,2*pi]
gridsteps = 19
bstep = np.pi/gridsteps
gstep = 2.0*np.pi/gridsteps
overall_best_z = []*nqubits
overall_best_Cz = 0.0
all_Cz = []
all_b = []
all_g = []
for b in range(0,gridsteps+1):
  for g in range(0,gridsteps+1):
    beta = bstep*b
    gamma = gstep*g
    all_b.append(b)
    all_g.append(g)

   #instantiate a circuit
    circuit = cirq.Circuit()
    #apply Hadamard gate to all qubits to define the initial state
    #here, the initial state is a uniform superposition over all basis states
    circuit.append(cirq.H(q) for q in qubits)

    #define operators for creating state |gamma,beta> = U_B*U_C*|s>
    #define U_C operator = Product_<jk> { e^(-i*gamma*C_jk)} and append to main circuit
    coeff = -0.5 #coefficient in front of sigma_z operators in the C_jk operator
    U_C = cirq.Circuit()
    for i in range(0,nqubits):
      U_C.append(cirq.CNOT.on(qubits[i],qubits[(i+1)%nqubits]))
      U_C.append(cirq.rz(2.0*coeff*gamma).on(qubits[(i+1)%nqubits]))
      U_C.append(cirq.CNOT.on(qubits[i],qubits[(i+1)%nqubits]))
    circuit.append(U_C)

    #define U_B operator = Product_j {e^(-i*beta*X_j)} and append to main circuit
    U_B = cirq.Circuit()
    for i in range(0,nqubits):
      U_B.append(cirq.H(qubits[i]))
      U_B.append(cirq.rz(2.0*beta).on(qubits[i]))
      U_B.append(cirq.H(qubits[i]))
    circuit.append(U_B)

    #add measurement operators for each qubit
    #measure in the computational basis to get the string z
    for i in range(0,nqubits):
      circuit.append(cirq.measure(qubits[i],key=str(i)))

    #run circuit in simulator to get the state |beta,gamma> and measure in the
    #computational basis to get the string z and evaluate C(z)
    #repeat for 100 runs and save the best z and C(z)
    #simulator = cirq.google.XmonSimulator()
    simulator = cirq.Simulator()
    reps = 100
    results = simulator.run(circuit, repetitions=reps)

    #get bit string z from results
    best_z = [None]*nqubits
    best_Cz = 0.0
    for i in range(0,reps):
      z = []
      for j in range (0,nqubits):
        if (results.measurements[str(j)][i]):
          z.append(1)
        else:
          z.append(-1)

      #compute C(z)
      Cz = 0.0
      for j in range(0,nqubits):
        Cz += 0.5*(-1.0*z[j]*z[(j+1)%nqubits] + 1.0)

      #store best values for z and C(z)
      if (Cz > best_Cz):
        best_Cz = Cz
        best_z = z
    all_Cz.append(best_Cz)
    if (best_Cz > overall_best_Cz):
      overall_best_Cz = best_Cz
      overall_best_z = best_z
      best_beta = beta
      best_gamma = gamma

#print best string z and corresponding C(z)
print("overall best z")
print(overall_best_z)
print("overall best Cz")
print(overall_best_Cz)
print("best beta")
print(best_beta)
print("best gamma")
print(best_gamma)

plt.plot(all_Cz)
plt.xlabel("Iteration number in Grid Search")
plt.ylabel("Best C(z) value")
plt.show()

#print a diagram of the circuit
print(circuit)
```

```python
from mpl_toolkits.mplot3d import Axes3D
from matplotlib import cm

fig = plt.figure()
ax = Axes3D(fig)
surf = ax.plot_trisurf(all_b, all_g, all_Cz, cmap=cm.jet, linewidth=0.1)
ax.set_xlabel('beta')
ax.set_ylabel('gamma')
ax.set_zlabel('best C(z) (out of 100 sims)')
fig.colorbar(surf, shrink=0.5, aspect=5)
plt.show()
```

```python

```

```python

```


Example 1
=========

Let us test ``cgn`` on a simple unconstrained linear least-squares problem. As an example, we use a regularized version of problem 32 from the article

More, J., Garbox, B. and Hillstrom, E. "Testing Unconstrained Optimization Software", 1981.

The problem is as follows:

$
\begin{align}
\min_{x \in \mathbb R^n} & \frac{1}{2}||F(x)||_2^2 + \frac{\beta}{2}||x||_2^2, \\
\text{where} \quad & F(x)_j = x_j - \frac{2}{m} \sum_{i=1}^n x_i - 1, \quad \text{for } 1 \leq j \leq n, \\
& F(x)_j = - \frac{2}{m} \sum_{i=1}^n x_i - 1, \quad \text{for } n < j <= m, \\
& \beta = \frac{1}{100},
\end{align}
$

with $m >= n$. Let us choose $n=200$ and $m=400$.

First, we implement the affine misfit function $F=F(x)$.

```python
import numpy as np

m = 400
n = 200

def F(x):
    z = np.zeros(m)
    for i in range(n):
        z[i] = x[i] - 2.0 * sum(x) / m - 1.
    for i in range(n,m):
        z[i] = - 2.0 * sum(x) / m - 1.
    return z
```

We also have to implement its Jacobian:

```python
A = np.zeros((m,n))
# upper half of the Jacobian matrix
for i in range(n):
    for j in range(n):
        if i == j:
            A[i,j] = 1.0 - 2.0 / m
        else:
            A[i,j] = -2.0 / m
# lower half of the Jacobian matrix
for i in  range(n,m):
    for j in range(n):
        A[i,j] = -2.0 / m

def DF(x):
    return A
```

Let us now set up the ``cgn.Parameter`` object. To create a ``cgn.Parameter`` object, the user has to provide a starting guess at initialization. This is necessary so that ``cgn`` can verify the consistency of the user input (e.g. that the dimensions match) before it starts solving the optimization problem. By doing that, it makes it easier to diagnose runtime errors that occur later. For our parameter $x$, we simply use the 1-vector as initial guess.

```python
import cgn

x = cgn.Parameter(start=np.ones(n), name="x")
```

Note that the starting guess can be adapted later via the ``x.start``-property.
Next, we set up the regularization term by setting $\beta = 1$. The regularization operator defaults to the identity, and the regularizing guess default to the zero vector.

```python
x.beta = 0.001
```

Next, we can initialize the ``cgn.Problem`` object:

```python
problem = cgn.Problem(parameters=[x], fun=F, jac=DF)
```

We initialize the solver...

```python
solver = cgn.CGN()
```

and solve the problem:

```python
x_start = np.ones(n)
solver.options.set_verbosity(2)
solution = solver.solve(problem=problem)
```

    Starting the constrained Gauss-Newton method. Cost at starting value: 500.1
    +-----------+-------------------------+-------------------------+-------------------------+-------------------------+-------------------------+
    | Iteration | Cost                    | Constraint violation    | Stepsize (||p||)        | Steplength (h)          | Computation time [s]    |
    +-----------+-------------------------+-------------------------+-------------------------+-------------------------+-------------------------+
    +-----------+-------------------------+-------------------------+-------------------------+-------------------------+-------------------------+
    |     1     |    100.09990009990057   |           0.0           |    28.270143239845737   |           1.0           |   0.012190103530883789  |
    +-----------+-------------------------+-------------------------+-------------------------+-------------------------+-------------------------+
    |     2     |    100.0999000999005    |           0.0           |   1.55683666436841e-13  |           0.25          |   0.03589987754821777   |
    +-----------+-------------------------+-------------------------+-------------------------+-------------------------+-------------------------+
    The iteration converged successfully after 2 steps.
    Minimum of cost function: 100.0999000999005

Let us view the solution:

```python
x_min = solution.minimizer("x")
x_min
```

    array([-0.999001, -0.999001, -0.999001, -0.999001, -0.999001, -0.999001,
           -0.999001, -0.999001, -0.999001, -0.999001, -0.999001, -0.999001,
           -0.999001, -0.999001, -0.999001, -0.999001, -0.999001, -0.999001,
           -0.999001, -0.999001, -0.999001, -0.999001, -0.999001, -0.999001,
           -0.999001, -0.999001, -0.999001, -0.999001, -0.999001, -0.999001,
           -0.999001, -0.999001, -0.999001, -0.999001, -0.999001, -0.999001,
           -0.999001, -0.999001, -0.999001, -0.999001, -0.999001, -0.999001,
           -0.999001, -0.999001, -0.999001, -0.999001, -0.999001, -0.999001,
           -0.999001, -0.999001, -0.999001, -0.999001, -0.999001, -0.999001,
           -0.999001, -0.999001, -0.999001, -0.999001, -0.999001, -0.999001,
           -0.999001, -0.999001, -0.999001, -0.999001, -0.999001, -0.999001,
           -0.999001, -0.999001, -0.999001, -0.999001, -0.999001, -0.999001,
           -0.999001, -0.999001, -0.999001, -0.999001, -0.999001, -0.999001,
           -0.999001, -0.999001, -0.999001, -0.999001, -0.999001, -0.999001,
           -0.999001, -0.999001, -0.999001, -0.999001, -0.999001, -0.999001,
           -0.999001, -0.999001, -0.999001, -0.999001, -0.999001, -0.999001,
           -0.999001, -0.999001, -0.999001, -0.999001, -0.999001, -0.999001,
           -0.999001, -0.999001, -0.999001, -0.999001, -0.999001, -0.999001,
           -0.999001, -0.999001, -0.999001, -0.999001, -0.999001, -0.999001,
           -0.999001, -0.999001, -0.999001, -0.999001, -0.999001, -0.999001,
           -0.999001, -0.999001, -0.999001, -0.999001, -0.999001, -0.999001,
           -0.999001, -0.999001, -0.999001, -0.999001, -0.999001, -0.999001,
           -0.999001, -0.999001, -0.999001, -0.999001, -0.999001, -0.999001,
           -0.999001, -0.999001, -0.999001, -0.999001, -0.999001, -0.999001,
           -0.999001, -0.999001, -0.999001, -0.999001, -0.999001, -0.999001,
           -0.999001, -0.999001, -0.999001, -0.999001, -0.999001, -0.999001,
           -0.999001, -0.999001, -0.999001, -0.999001, -0.999001, -0.999001,
           -0.999001, -0.999001, -0.999001, -0.999001, -0.999001, -0.999001,
           -0.999001, -0.999001, -0.999001, -0.999001, -0.999001, -0.999001,
           -0.999001, -0.999001, -0.999001, -0.999001, -0.999001, -0.999001,
           -0.999001, -0.999001, -0.999001, -0.999001, -0.999001, -0.999001,
           -0.999001, -0.999001, -0.999001, -0.999001, -0.999001, -0.999001,
           -0.999001, -0.999001, -0.999001, -0.999001, -0.999001, -0.999001,
           -0.999001, -0.999001])

Let us compare this solution to the one obtained with the implementation of ridge regression from ``scikit-learn``:

```python
from sklearn.linear_model import Ridge
clf = Ridge(alpha=x.beta, fit_intercept=False)
y = np.ones(m)
clf.fit(X=A, y=y)
x_ridge = clf.coef_

difference_to_ridge = np.linalg.norm(x_min - x_ridge)
print(f"Difference to ridge: {difference_to_ridge}")
```

    Difference to ridge: 3.467185801042848e-14

The two solutions agree up to a precision of $10^{-13}$!

Note that the ``solution`` object also provides us access to the minimum of the cost function. Here, it is important to keep in mind that the cost function always has a factor $\frac{1}{2}$ in front.

```python
cost_at_minimum = solution.cost
print(f"Cost at minimum: {cost_at_minimum}")
```

    Cost at minimum: 100.0999000999005

Let us verify this by manually recomputing the cost at ``x_min``:

```python
cost_recomputed = 0.5 * np.sum(np.square(F(x_min))) + 0.5 * x.beta * np.sum(np.square(x_min))
print(f"Cost at minimum, recomputed: {cost_at_minimum}")
```

    Cost at minimum, recomputed: 100.0999000999005

And indeed, both numbers agree.

Finally, the ``solution`` also provides access to the [precision matrix](https://en.wikipedia.org/wiki/Precision_(statistics)) via ``solution.precision``.

```python
precision = solution.precision
precision.shape
```

    (200, 200)

This is relevant in the case where our optimization problem comes from [maximum likelihood estimation](https://en.wikipedia.org/wiki/Maximum_likelihood_estimation) or
[Bayesian maximum-a-posteriori estimation](https://en.wikipedia.org/wiki/Maximum_a_posteriori_estimation), and the cost function actually corresponds to a negative log-likelihood or a negative log-posterior density, respectively.


```
import sympy as smp
x, y, t = smp.symbols('x, y, t')
import numpy as np
pi = np.pi
```

Define an abribrary solution.

```
#u = smp.sin(0.5*pi*t) * smp.cos(3*pi*(x-1.0)) * smp.cos(7*pi*y)
u = smp.cos(0.5*pi*t) * smp.cos(3*pi*(x-1.0)) * smp.cos(7*pi*y)
#u = (1-smp.cos(0.5*pi*t)) * smp.cos(3*pi*(x-1.0)) * smp.cos(7*pi*y)
#u = smp.log(1+t) * smp.cos(3*pi*(x-1.0)) * smp.cos(7*pi*y)
```

Define matching right-hand side for heat equation $u_t - \Delta u = f$.

```
f = smp.diff(u,t) - smp.diff(smp.diff(u, x), x) - smp.diff(smp.diff(u, y), y)
f = smp.expand(f)
```

Define forward Euler scheme:

```
def forward_euler(u0, dt, f0):
    u1 = u0 + dt * (smp.diff(u0, x, 2) + smp.diff(u0, y, 2)) + dt * f0
    #u1 = u0
    return u1
```

Perform one step with the above solution.

```
Dt = [0.5**k for k in range(10)]
Err = []
for dt in Dt:
    u0 = u.subs({t:0.0})
    f0 = f.subs({t:0.0})
    u_approx = forward_euler(u0, dt, f0)
    u1 = u.subs({t:dt})
    err = u1 - u_approx
    # Compute L^2 norm over unit square [0,1]^2.
    err_norm = smp.sqrt(smp.integrate(err**2, (x, 0.0, 1.0), (y, 0.0, 1.0)))
    Err.append(err_norm)
```

Plot error norm.

```
from matplotlib import pyplot as pp
# Show orders 1,2,3 for comparison
for o in [1,2,3,4]:
    pp.loglog([Dt[0], Dt[-1]], [Err[0], Err[0]*(Dt[-1]/Dt[0])**o], color='0.5')
pp.loglog(Dt, Err,'o-')
pp.show()
```

```

```

```

```

test/Order of convergence of the forward Euler scheme.ipynb