# Whole-body manipulation
The objective of this exercise is to reach multiple targets while keeping balance in the Talos humanoid robot.

This exercise focuses on a multi-contact optimal control problem of the form:

\begin{equation}\nonumber
	\begin{aligned}
		\min_{\mathbf{x}_s,\mathbf{u}_s}
		&\hspace{-2.em}
		& & \hspace{-0.75em}l_N(\mathbf{x}_{N})+\sum_{k=0}^{N-1} \int_{t_k}^{t_k+\Delta t_k}\hspace{-2.em} l_k(\mathbf{x}_k,\mathbf{u}_k)dt \hspace{-8.em}&\\
		& \hspace{-1em}\textrm{s.t.}
		& & \mathbf{q}_{k+1} = \mathbf{q}_k \oplus \int_{t_k}^{t_k+\Delta t_k}\hspace{-2.em}\mathbf{v}_{k+1}\,dt, &\textrm{(integrator)}\\
		& & & \mathbf{v}_{k+1} = \mathbf{v}_k + \int_{t_k}^{t_k+\Delta t_k}\hspace{-2.em}\mathbf{\dot{v}}_k\,dt, &\\
		& & & \hspace{-1em}\left[\begin{matrix}\mathbf{\dot{v}}_k \\ -\boldsymbol{\lambda}_k\end{matrix}\right] =
		\left[\begin{matrix}\mathbf{M} & \mathbf{J}^{\top}_c \\ {\mathbf{J}_{c}} & \mathbf{0} \end{matrix}\right]^{-1}
		\left[\begin{matrix}\boldsymbol{\tau}_b \\ -\mathbf{a}_0 \\\end{matrix}\right], &\textrm{(contact dynamics)}\\
		& & & \mathbf{R}\boldsymbol{\lambda}_{\mathcal{C}(k)} \leq \mathbf{\mathbf{r}}, &\textrm{(friction-cone)}\\
			& & & \mathbf{\bar{x}} \leq \mathbf{x}_k \leq \mathbf{\underline{x}}, &\textrm{(state bounds)}
		\end{aligned}
\end{equation}}

where $l_i(\mathbf{x}_i, \mathbf{u}_i) = w_{hand}\|\log{(\mathbf{p}_{\mathcal{G}(k)}(\mathbf{q}_k)^{-1} \mathbf{^oM}_{\mathbf{f}_{\mathcal{G}(k)}})}\| + w_{xreg}\|\mathbf{x} - \mathbf{x}_0\|_{Q} + w_{ureg}\|\mathbf{u}\|_{R}$. Note that (1) the first term is the hand placement cost and (2) the terminal cost does not include the control regularization term.

Below there is a basic example that defines the above problem for reaching one target. Later, you will have to build the problem on top of it.

Without no more preamble, let's first declare the robot model and the foot and hand names!

```python
import crocoddyl
import example_robot_data
import numpy as np
import pinocchio as pin

# Load robot
robot = example_robot_data.load('talos')
rmodel = robot.model
q0 = rmodel.referenceConfigurations["half_sitting"]
x0 = np.concatenate([q0, np.zeros(rmodel.nv)])

# Declaring the foot and hand names
rf_name = "right_sole_link"
lf_name = "left_sole_link"
lh_name = "gripper_left_joint"

# Getting the frame ids
rf_id = rmodel.getFrameId(rf_name)
lf_id = rmodel.getFrameId(lf_name)
lh_id = rmodel.getFrameId(lh_name)

# Define the robot's state and actuation
state = crocoddyl.StateMultibody(rmodel)
actuation = crocoddyl.ActuationModelFloatingBase(state)
```

With the following function, we could build a differential action model giving a desired hand target.
The function builds a double-support contact phase and defines a hand-placement task. The cost function also includes:
 - state and control regularization terms
 - state limits penalization
 - friction cone penalization

```python
def createActionModel(target):
    # Creating a double-support contact (feet support)
    contacts = crocoddyl.ContactModelMultiple(state, actuation.nu)
    lf_contact = crocoddyl.ContactModel6D(state, lf_id, pin.SE3.Identity(), actuation.nu, np.array([0, 0]))
    rf_contact = crocoddyl.ContactModel6D(state, rf_id, pin.SE3.Identity(), actuation.nu, np.array([0, 0]))
    contacts.addContact("lf_contact", lf_contact)
    contacts.addContact("rf_contact", rf_contact)

    # Define the cost sum (cost manager)
    costs = crocoddyl.CostModelSum(state, actuation.nu)

    # Adding the hand-placement cost
    w_hand = np.array([1] * 3 + [0.0001] * 3)
    lh_Mref = pin.SE3(np.eye(3), target)
    activation_hand = crocoddyl.ActivationModelWeightedQuad(w_hand**2)
    lh_cost = crocoddyl.CostModelFramePlacement(state, activation_hand, lh_id, lh_Mref, actuation.nu)
    costs.addCost("lh_goal", lh_cost, 1e2)

    # Adding state and control regularization terms
    w_x = np.array([0] * 3 + [10.] * 3 + [0.01] * (state.nv - 6) + [10] * state.nv)
    activation_xreg = crocoddyl.ActivationModelWeightedQuad(w_x**2)
    x_reg_cost = crocoddyl.CostModelState(state, activation_xreg, x0, actuation.nu)
    u_reg_cost = crocoddyl.CostModelControl(state, actuation.nu)
    costs.addCost("xReg", x_reg_cost, 1e-3)
    costs.addCost("uReg", u_reg_cost, 1e-4)

    # Adding the state limits penalization
    x_lb = np.concatenate([state.lb[1:state.nv + 1], state.lb[-state.nv:]])
    x_ub = np.concatenate([state.ub[1:state.nv + 1], state.ub[-state.nv:]])
    activation_xbounds = crocoddyl.ActivationModelQuadraticBarrier(crocoddyl.ActivationBounds(x_lb, x_ub))
    x_bounds = crocoddyl.CostModelState(state, activation_xbounds, 0 * x0, actuation.nu)
    costs.addCost("xBounds", x_bounds, 1.)

    # Adding the friction cone penalization
    nsurf, mu = np.array([0, 0, 1]), 0.7
    cone = crocoddyl.FrictionCone(nsurf, mu, 4, False)
    activation_friction = crocoddyl.ActivationModelQuadraticBarrier(crocoddyl.ActivationBounds(cone.lb, cone.ub))
    lf_friction = crocoddyl.CostModelContactFrictionCone(state, activation_friction, lf_id, cone, actuation.nu)
    rf_friction = crocoddyl.CostModelContactFrictionCone(state, activation_friction, rf_id, cone, actuation.nu)
    costs.addCost("lf_friction", lf_friction, 1e1)
    costs.addCost("rf_friction", rf_friction, 1e1)

    # Creating the action model
    dmodel = crocoddyl.DifferentialActionModelContactFwdDynamics(state, actuation, contacts, costs)
    return dmodel
```

And to easily build a sequence of tasks, we have the following function

```python
def createSequence(dmodels, DT, N):
    return [[crocoddyl.IntegratedActionModelEuler(m, DT)] * N +
            [crocoddyl.IntegratedActionModelEuler(m, 0.)] for m in dmodels]
```

Finally, the following function allows us to display the motions and desired targets:

```python
import meshcat.geometry as g
import meshcat.transformations as tf

def createDisplay(targets):
    display = crocoddyl.MeshcatDisplay(robot, 4, 4, False)
    for i, target in enumerate(targets):
        display.robot.viewer["target_" + str(i)].set_object(g.Sphere(0.05))
        Href = np.array([[1., 0., 0., target[0]],
                         [0., 1., 0., target[1]],
                         [0., 0., 1., target[2]],
                         [0., 0., 0., 1.]])
        display.robot.viewer["target_" + str(i)].set_transform(np.array([[1., 0., 0., target[0]],
                         [0., 1., 0., target[1]],
                         [0., 0., 1., target[2]],
                         [0., 0., 0., 1.]]))
    return display
```

Now, we create an optimal control problem to reach a single target

```python
DT, N = 5e-2, 20
target = np.array([0.4, 0, 1.2])

# Creating a running model for the target
dmodel = createActionModel(target)
seqs = createSequence([dmodel], DT, N)

# Defining the problem and the solver
problem = crocoddyl.ShootingProblem(x0, sum(seqs, [])[:-1], seqs[-1][-1])
fddp = crocoddyl.SolverFDDP(problem)

# Creating display
display = createDisplay([target])

# Adding callbacks to inspect the evolution of the solver (logs are printed in the terminal)
fddp.setCallbacks([crocoddyl.CallbackVerbose(), crocoddyl.CallbackDisplay(display)])

# Embedded in this cell
display.robot.viewer.jupyter_cell()
```

Let's solve this problem!

```python
print("Problem solved:", fddp.solve())
print("Number of iterations:", fddp.iter)
print("Total cost:", fddp.cost)
print("Gradient norm:", fddp.stoppingCriteria())
```

You could display again the final solution

```python
display.rate = -1
display.freq = 1
display.displayFromSolver(fddp)
```

## Modifying the example

Let's build an optimal control problem to reach 4 targets as described below:

```python
targets = []
targets += [np.array([0.4, 0.1, 1.2])]
targets += [np.array([0.6, 0.1, 1.2])]
targets += [np.array([0.6, -0.1, 1.2])]
targets += [np.array([0.4, -0.1, 1.2])]
```

Now let's display the targets in Meshcat. Do not forget to embed again the display into the jupyter cell

```python
display = createDisplay(targets)

# Embedded in this cell
display.robot.viewer.jupyter_cell()
```

After checking that everything is alright, it's time to build the sequence!
Do not forget to create the problem as well :)

And we solve it as before

```python
# Create the FDDP solver
fddp = crocoddyl.SolverFDDP(problem)
fddp.setCallbacks([crocoddyl.CallbackVerbose(), crocoddyl.CallbackDisplay(display)])

# Solves the problem
print("Problem solved:", fddp.solve())
print("Number of iterations:", fddp.iter)
print("Total cost:", fddp.cost)
print("Gradient norm:", fddp.stoppingCriteria())
```

Do not miss the change to display the motion at the right display speed!

```python
display.rate = -1
display.freq = 1
display.displayFromSolver(fddp)
```

## Same targets with right hand

You've learned how to reach 4 targets with the left hand, congratulations!

To keep playing within this problem, you should create a new createActionModel to achieve the same task for the right hand.

```python
def createActionModel(target):
    # now god is with you xD
    # time to show you up!
```

And here you need to create the problem and solve.
Do not forget to display the results

```python

```


```python
import numpy as np
import pandas as pd
import linearsolve as ls
import matplotlib.pyplot as plt
plt.style.use('classic')
%matplotlib inline
```

# Homework 6

**Instructions:** Complete the notebook below. Download the completed notebook in HTML format. Upload assignment using Canvas.

**Due:** Feb. 16 at **2pm.**

## Exercise: Four-Period Cake-Eating Problem

Now consider a the four-period cake-eating problem. A person has an initial quantity of cake $K_0>0$ and receives utility from consuming cake in period 0, period 1, period 2, and period 3:

\begin{align}
U(C_0,C_1,C_2,C_3) & = \log C_0 + \beta \log C_1+ \beta^2 \log C_2 + \beta^3 \log C_3.
\end{align}

The person chooses how much to consume in each period, $C_0$, $C_1$, $C_2$, $C_3$, and quantities of cake $K_1$, $K_2$, and $K_3$ to carry foward from period 0, period 1, and period 2 to maximize her utility subject to the following two budget constraints:

\begin{align}
C_0 + K_1 & = K_0\\
C_1 + K_2 & = K_1\\
C_2 + K_3 & = K_2\\
C_3 & = K_3,
\end{align}

where the boundary condition $K_4 = 0$ has been imposed.

### Part (a)

Find the first-order conditions with respect to $K_1$, $K_2$, and $K_3$ for the four-period cake-eating problem. Then solve by hand for:

1. $K_3$ as a function of $K_2$
1. $K_2$ as a function of $K_1$
2. $K_1$ as a function of $K_0$

Use what you derive in this part to do **Part (b)**, **Part (c)**, and **Part (d)**.

### Part (b)

Suppose that the initial quantity of cake is $K_0 = 100$. For $\beta = 0, 0.25, 0.5, 0.75, 0.99, 1, 2$, plot the optimal path for the quantity of cake for periods 0 through 4. Add labels to the plot command so you know which line corresponds to which value of $\beta$.

```python
# Define a function to compute the solution for the four-period cake-eating problem. CELL NOT PROVIDED
def four_period_cake(beta,cake0):

    '''Function for computing solution to four-period cake-eating problem

            Args:
            beta (float):   subjective discount factor
            cake0 (float):  initial quantity of cake

        Returns:
            numpy array, numpy array
    '''

    # Initialize arrays for cake and consumption. NOT PROVIDED
    cake = np.zeros(5)
    consumption = np.zeros(4)

    # Assign values to cake array elements. NOT PROVIDED
    cake[0] = cake0
    cake[1] = (beta+beta**2+beta**3)/(1+beta+beta**2+beta**3)*cake[0]
    cake[2] = (beta+beta**2)/(1+beta+beta**2)*cake[1]
    cake[3] = beta/(1+beta)*cake[2]

    # Assign values to consumption array elements as implied by the budget constraints. NOT PROVIDED
    consumption[0] = cake[0]-cake[1]
    consumption[1] = cake[1]-cake[2]
    consumption[2] = cake[2]-cake[3]
    consumption[3] = cake[3]

    return cake,consumption
```

```python
# Create a variable called 'cake0' that stores the initial quantity of cake. NOT PROVIDED
cake0=100

# Plot optimal path of cake for the different values of beta. NOT PROVIDED
for beta in [2,1, 0.99, 0.75, 0.5, 0.25, 0]:

    cake,consumption = four_period_cake(beta=beta,cake0=cake0)
    plt.plot(cake,'o-',lw=3,alpha=0.75,label='$\\beta=$'+str(beta))

plt.title('Cake Quantity for different $\\beta$ values')

# Put a legend to the right of the current axis. NOT PROVIDED
plt.legend(loc='center left', bbox_to_anchor=(1, 0.5))

plt.grid()
```

### Part (c)

Suppose that the initial quantity of cake is $K_0 = 100$. For $\beta = 0, 0.25, 0.5, 0.75, 0.99, 1, 2$, plot the optimal path for cake consumption for periods 0 through 3. Add labels to the plot command so you know which line corresponds to which value of $\beta$.

```python
# Create a variable called 'cake0' that stores the initial quantity of cake. NOT PROVIDED
cake0=100

# Plot optimal path of cake for the different values of beta. NOT PROVIDED
for beta in [2,1, 0.99, 0.75, 0.5, 0.25, 0]:

    cake,consumption = four_period_cake(beta=beta,cake0=cake0)
    plt.plot(consumption,'o-',lw=3,alpha=0.75,label='$\\beta=$'+str(beta))

plt.title('Cake Consumption for different $\\beta$ values')

# Put a legend to the right of the current axis. NOT PROVIDED
plt.legend(loc='center left', bbox_to_anchor=(1, 0.5))

plt.grid()
```

### Part (d)

Suppose that $\beta=0.95$. For $K_0 = 1, 10, 20, 50$, plot the optimal path for the quantity of cake for periods 0 through 4. Add labels to the plot command so you know which line corresponds to which value of $K_0$.

```python
# Create a variable called 'cake0' that stores the initial quantity of cake. NOT PROVIDED
beta=0.95

# Plot optimal path of cake for the different values of beta. NOT PROVIDED
for cake0 in [1,10,20,50]:

    cake,consumption = four_period_cake(beta=beta,cake0=cake0)
    plt.plot(cake,'o-',lw=3,alpha=0.75,label='$K_0=$'+str(cake0))

plt.title('Cake Quantity for different $K_0$ values')

# Put a legend to the right of the current axis. NOT PROVIDED
plt.legend(loc='center left', bbox_to_anchor=(1, 0.5))

plt.grid()
```

## Exercise: The Stochastic Solow Growth Model

In class we simulated the stochastic Solow growth model with three endogenous variables: $K_t$, $Y_t$, and $A_t$. Now, reconsider the model with two additional variables, consumption $C_t$ and investment $I_t$, and two additional equations:

\begin{align}
Y_t & = A_t K_t^{\alpha}\\
C_{t} & = (1-s)Y_t\\
Y_{t} & = C_t + I_t\\
K_{t+1} & = sY_t + (1-\delta) K_t\\
\log A_{t+1} & = \rho \log A_t + \epsilon_{t+1}
\end{align}

where $\epsilon_{t+1} \sim \mathcal{N}(0,\sigma^2)$. In this exercise, you will construct a 401 period (10-year) simulation of the model.

For the other parameter, use the following values for the simulation:

| $$\rho$$ | $$\sigma$$ | $$s$$  | $$\alpha$$ | $$\delta $$ | $$T$$  |
|----------|------------|--------|------------|-------------|--------|
| 0.8      | 0.005      | 0.12   | 0.33       |  0.025      | 401    |

```python
# Create a variable called 'parameters' that is a Pandas Series containing the parameter values for the simulation
parameters = pd.Series(dtype=float)
parameters['rhoa'] = .8
parameters['sigma_squared'] = 0.005**2
parameters['alpha'] = 0.33
parameters['delta'] = 0.025
parameters['s'] = 0.12

# Print the model's parameters
print(parameters)
```

    rhoa             0.800000
    sigma_squared    0.000025
    alpha            0.330000
    delta            0.025000
    s                0.120000
    dtype: float64

```python
# Define a function that evaluates the equilibrium conditions. PROVIDED
def equilibrium_equations(variables_forward,variables_current,parameters):

    # Parameters. PROVIDED
    p = parameters

    # Current variables. PROVIDED
    cur = variables_current

    # Forward variables. PROVIDED
    fwd = variables_forward

    # Production function
    prod_fn = cur.a*cur.k**p.alpha - cur.y

    # Capital evolution
    capital_evolution = p.s*cur.a*cur.k**p.alpha + (1 - p.delta)*cur.k - fwd.k

    # Consumption function
    consumption_function = (1-p.s)*cur.y - cur.c

    # Market clearing
    market_clearing = cur.c + cur.i - cur.y

    # Exogenous tfp
    tfp_proc = p.rhoa*np.log(cur.a) - np.log(fwd.a)

    # Stack equilibrium conditions into a numpy array
    return np.array([
        prod_fn,
        capital_evolution,
        consumption_function,
        market_clearing,
        tfp_proc
        ])
```

```python
# Initialize the model. Note that state variables must be named first in the var_names argument and shock for each
# state variable must be named.
model = ls.model(equations = equilibrium_equations,
                 n_states=2,
                 var_names=['a','k','y','c','i'],
                 shock_names=['e_a','e_k'],
                 parameters = parameters)
```

```python
# Compute the steady state numerically using the .compute_ss() method.
guess = [1,4,1,1,1]
model.compute_ss(guess)
```

```python
# Use the .approximate_and_solve() method to compute the log-linear approximation of the model around the
# non-stochastic steady state and solve endogenous variables in terms of state variables.
model.approximate_and_solve()
```

```python
# Use .stoch_sim() method to compute stochastic simulation with the following arguments:
#    1. seed = 126
#    2. T = 401
#    3. cov_mat= [[parameters['sigma_squared'],0],[0,0]]
model.stoch_sim(seed=126,T=401,cov_mat= [[parameters['sigma_squared'],0],[0,0]])

# Print first 10 rows of model.simulated
print(model.simulated.head(10))
```

            e_a  e_k         a         k         y         c         i
    0  0.001237  0.0  0.004041  0.000614  0.004243  0.004243  0.004243
    1  0.001189  0.0  0.004421  0.000705  0.004654  0.004654  0.004654
    2  0.002629  0.0  0.006166  0.000803  0.006431  0.006431  0.006431
    3  0.009421  0.0  0.014354  0.000944  0.014665  0.014665  0.014665
    4 -0.001482  0.0  0.010001  0.001287  0.010426  0.010426  0.010426
    5 -0.011009  0.0 -0.003008  0.001516 -0.002508 -0.002508 -0.002508
    6  0.003379  0.0  0.000972  0.001415  0.001439  0.001439  0.001439
    7  0.005128  0.0  0.005905  0.001416  0.006373  0.006373  0.006373
    8 -0.001113  0.0  0.003611  0.001540  0.004119  0.004119  0.004119
    9 -0.005668  0.0 -0.002779  0.001604 -0.002249 -0.002249 -0.002249

```python
# Plot the computed stochastic simulation results. Creae two axes side-by-side.
#     1. Left axes: plot: a, k, i, y, c
#     1. Right axes: plot: shock to TFP
fig = plt.figure(figsize=(12,4))

ax1 = fig.add_subplot(1,2,1)
(model.simulated[['a','y','k','c','i']]*100).plot(lw=3,alpha=0.5,grid=True,ax = ax1).legend(loc='upper right',ncol=3)
ax1.set_title('Endogenous Variables')
ax1.set_xlabel('% dev. from steady state')

ax2 = fig.add_subplot(1,2,2)
(model.simulated[['e_a']]*100).plot(lw=3,alpha=0.5,grid=True,ax = ax2).legend(loc='upper right',ncol=2)
ax2.set_title('TFP Shock Process')
ax2.set_xlabel('% dev. from steady state')

fig.tight_layout()
```

```python
# Compute standard deviations of simulated TFP, output, consumption, investment, and capital
model.simulated[['a','y','c','i','k']].std()
```

    a    0.008863
    y    0.009033
    c    0.009033
    i    0.009033
    k    0.002546
    dtype: float64

```python
# Compute correlation coefficients of simulated TFP, output, consumption, investment, and capital
model.simulated[['a','y','c','i','k']].corr()
```

      a
      y
      c
      i
      k

      a
      1.000000
      0.995770
      0.995770
      0.995770
      0.156264

      y
      0.995770
      1.000000
      1.000000
      1.000000
      0.246354

      c
      0.995770
      1.000000
      1.000000
      1.000000
      0.246354

      i
      0.995770
      1.000000
      1.000000
      1.000000
      0.246354

      k
      0.156264
      0.246354
      0.246354
      0.246354
      1.000000

**Questions**

1. In the data simulated from the stochastic Solow growth model, which variables appear to be perfectly correlated?
2. Refer to the Notebook from Class 9. Compared with the actual correlations, does the stochastic Solow growth model under- or overpredict the correlation between output and consumption over the cycle? What about the correlation between output and investment?

**Answers**

1. Output, investment, and consumption appear to be perfectly correlated.
2. The stochastic Solow model *overpredicts* the correlation between output and consumption. The stochastic Solow model also *overpredicts* the correlation between output and investment.


## 4.2 常用数学符号

常用数学符号包括运算符号、标记符号、各类括号、空心符号及一些特殊函数。

### 4.2.1 运算符号

在初等数学中，最基本的运算规则是加减乘除。在LaTeX中，加法符号和减法符号就是+和-；而乘法符号有两种，第一种是`\times`，对应于符号$\times$，第二种是`\cdot`，对应于符号$\cdot$，除法符号的命令为`\div`。

【**例4-15**】书写加减乘除$3+5$、$3-5$、$3\times 5$、$3\cdot 5$、$3\div 5$和$3/5$。

```tex
\documentclass[12pt]{article}
\begin{document}

$$3+5$$        % 加法
$$3-5$$        % 减法
$$3\times 5$$  % 乘法
$$3\cdot 5$$   % 乘法
$$3\div 5$$    % 除法
$$3/5$$        % 除法

\end{document}
```

在加减的基础上，命令`\pm`（由plus和minus首字母构成）和`\mp`（由minus和plus首字母构成）分别对应着符号$\pm$和$\mp$。与加减乘除同样常用的运算符号还有大于号、小于号等。

【**例4-16**】书写$x<y$、$x>y$、$x\leq y$、$x\geq y$、$x\ll y$和$x\gg y$。

```tex
\documentclass[12pt]{article}
\begin{document}

$$x<y$$      % 小于
$$x>y$$      % 大于
$$x\leq y$$  % 小于或等于
$$x\geq y$$  % 大于或等于
$$x\ll y$$   % 远小于
$$x\gg y$$   % 远大于

\end{document}
```

对于集合而言，还有一些基本运算符号如$\cap$（`\cap`）、$\cup$（`\cup`）、$\supset$（`\supset`）、$\subset$（`\subset`）、$\supseteq$（`\supseteq`）、$\in$（`\in`）。除此之外，与“属于”命令`\in`相反的“不属于”命令为`\notin`，编译效果为$\notin$。

### 4.2.2 标记符号

在数学中，还有一些基本数学表达式也非常重要，例如分式、上标、下标。LaTeX中用于书写分数和分式的基本命令为`\frac{分子}{分母}`，根据场景需要，也可以选用`\dfrac{分子}{分母}`和`\tfrac{分子}{分母}`。

【**例4-17**】书写分数$\frac{3}{5}$与分式$\frac{x}{y}$、$\frac{x+3}{y+5}$。

```tex
\documentclass[12pt]{article}
\begin{document}

$$\frac{3}{5}$$      % 分数
$$\frac{x}{y}$$      % 分式1
$$\frac{x+3}{y+5}$$  % 分式2

\end{document}
```

上标和下标在各类数学表达式中是非常常见的。

【**例4-18**】书写带上标的$x^{3}$、$x^{5}$、$x^{x+5}$和$x^{x^{3}+5}$。

```tex
\documentclass[12pt]{article}
\begin{document}

$$x^{3}$$
$$x^{5}$$
$$x^{x+5}$$
$$x^{x^{3}+5}$$

\end{document}
```

【**例4-19**】书写带下标的$x_{3}$、$x_{5}$、$x_{x+5}$、$x_{x_{3}+5}$和$x_{1},x_{2},\ldots,x_{n}$。

```tex
\documentclass[12pt]{article}
\begin{document}

$$x_{3}$$
$$x_{5}$$
$$x_{x+5}$$
$$x_{x_{3}+5}$$
$$x_{1},x_{2},\ldots,x_{n}$$

\end{document}
```

与上标和下标对应的还有各种“帽子”符号，即字母上面加符号。

【**例4-20**】书写$\hat{x}$、$\bar{x}$、$\tilde{x}$、$\vec{x}$和$\dot{x}$。

```tex
\documentclass[12pt]{article}
\begin{document}

$$\hat{x}$$
$$\bar{x}$$
$$\tilde{x}$$
$$\vec{x}$$
$$\dot{x}$$

\end{document}
```

与`\bar`作用相同的命令是`\overline`，不过`\overline`编译出来的“帽子”会比`\bar`宽一点，例如`\overline{xy}`可以编译出$\overline{xy}$，相比之下，`\bar{xy}`的编译效果为$\bar{xy}$。如果想要编译出比`\tilde`更宽一点的波浪线“帽子”，可以使用`\widetilde`，例如`\widetilde{xy}`可以编译出$\widetilde{xy}$，相比之下，`\tilde{xy}`的编译效果为$\tilde{xy}$。

> 参考[Tilde over the letter](https://latex.org/forum/viewtopic.php?f=48&t=11388)

根号同样是数学表达式中的常见符号，在LaTeX中，根号的命令为`\sqrt{}`，使用默认设置，生成的表达式为二次方根，如果想要设置为三次方根，则需要调整默认设置，即`\sqrt[3]{}`，以此类推，可以设置四次方根等。

【**例4-21**】书写根式$\sqrt{3}$、$\sqrt[3]{5}$、$\sqrt{x+y}$、$\sqrt{x^{3}+y^{5}}$、$\sqrt{1+\sqrt{x}}$。

```tex
\documentclass[12pt]{article}
\begin{document}

$$\sqrt{3}$$
$$\sqrt[3]{5}$$
$$\sqrt{x+y}$$
$$\sqrt{x^{3}+y^{5}}$$
$$\sqrt{1+\sqrt{x}}$$

\end{document}
```

【**例4-22**】书写分式和根式组合的表达式$\frac{\sqrt{x+3}}{\sqrt{y+5}}$。

```tex
\documentclass[12pt]{article}
\begin{document}

$$\frac{\sqrt{x+3}}{\sqrt{y+5}}$$

\end{document}
```

### 4.2.3 各类括号

在数学表达式中，括号的用处和种类都非常多，例如最常见的小括号、中括号、大括号（即花括号）。

【**例4-23**】书写数学表达式$x\left(\frac{1}{y}+1\right)$、$x\left[\frac{1}{y}+1\right]$和$x\left\{\frac{1}{y}+1\right\}$。

```tex
\documentclass[12pt]{article}
\begin{document}

$$x\left(\frac{1}{y}+1\right)$$
$$x\left[\frac{1}{y}+1\right]$$
$$x\left\{\frac{1}{y}+1\right\}$$

\end{document}
```
有时候，由于公式过长等原因，我们也可以在需要分行处插入`\\`将括号内的公式切分成多行。

> 参考[Bracket size proble: Split multi-line equations](https://latex.org/forum/viewtopic.php?f=46&t=4823)

【**例4-24**】运用双斜杠`\\`将括号内的公式切分成多行.

\begin{equation}
\begin{aligned}
\Bigl(a+\frac{b}{2}+\frac{c}{3}++\frac{d}{4}+\frac{e}{5}++\frac{f}{6}++\frac{g}{7}++\frac{h}{8} \\
      +\frac{i}{9}+\frac{j}{10}+\frac{k}{11}++\frac{l}{12}+\frac{m}{13}++\frac{n}{14}++\frac{o}{15}+\cdots\Bigr)
\end{aligned}
\end{equation}

```tex
\documentclass[12pt]{article}
\usepackage{amsmath}

\begin{document}

\begin{equation}
\begin{aligned}
\Bigl(a+\frac{b}{2}+\frac{c}{3}++\frac{d}{4}+\frac{e}{5}++\frac{f}{6}++\frac{g}{7}++\frac{h}{8} \\
      +\frac{i}{9}+\frac{j}{10}+\frac{k}{11}++\frac{l}{12}+\frac{m}{13}++\frac{n}{14}++\frac{o}{15}+\cdots\Bigr)
\end{aligned}
\end{equation}

\end{document}
```

在这里，我们可以使用一系列命令代替最常用的`\left`和`\right`组合，如`\bigl`和`\bigr`组合、`\Bigl`和`\Bigr`组合、`\biggl`和`\biggr`组合、`\Biggl`和`\Biggr`组合来控制括号大小。

【**例4-25**】运用`\bigl`和`\bigr`组合、`\Bigl`和`\Bigr`组合、`\biggl`和`\biggr`组合、`\Biggl`和`\Biggr`组合来控制括号大小.

```tex
\documentclass[12pt]{article}
\usepackage{amsmath}

\begin{document}

\begin{equation}

\left(x+y=z \right)
\bigl(x+y=z \bigr)
\Bigl(x+y=z \Bigr)
\biggl(x+y=z \biggr)
\Biggl(x+y=z \Biggr)

\end{equation}

\end{document}
```

编译上述代码，得到幻灯片如图4.2.11所示。

图4.2.11 编译后的幻灯片效果

在数学公式编辑中，除了以上常见的括号，也有一些广义的“括号”。

【**例4-26**】书写数学表达式$x\left|\frac{1}{y}+1\right|$、$x\left\|\frac{1}{y}+1\right\|$、$\left<\frac{1}{x},\frac{1}{y}\right>$和$\langle\frac{1}{x},\frac{1}{y}\rangle$。

```tex
\documentclass[12pt]{article}
\begin{document}

$$x\left|\frac{1}{y}+1\right|$$
$$x\left\|\frac{1}{y}+1\right\|$$
$$\left<\frac{1}{x},\frac{1}{y}\right>$$
$$\langle\frac{1}{x},\frac{1}{y}\rangle$$

\end{document}
```

在这里，使用半边括号，也能书写出导数的表达式。

【**例4-27**】书写导数$\left.\frac{dy}{dx}\right|_{x=1}$。

```tex
\documentclass[12pt]{article}
\begin{document}

$$\left.\frac{dy}{dx}\right|_{x=1}$$

\end{document}
```

### 4.2.4 空心符号

在数学表达式中，我们有时候会用到一些约定俗成的空心符号表示集合，这包括：

- 空心R符号$\mathbb{R}$表示由所有实数构成的集合。
- 空心Z符号$\mathbb{Z}$表示由所有整数构成的集合。
- 空心N符号$\mathbb{N}$表示由所有非负整数构成的集合，如果要表示正整数，使用符号$\mathbb{N}_{+}$即可。
- 空心C符号$\mathbb{C}$表示由所有复数构成的集合。

需要注意的是，要想让LaTeX成功编译出这些空心符号，我们需要调用特定的工具包，即`\usepackage{amsfonts}`，一般而言，为了保证公式的编译不出现意外，还会用到其他工具包，即`\usepackage{amsfonts, amssymb, amsmath}`。当然，除了这些，还有许多其他符号，这里不再一一赘述。

【**例4-28**】书写表达式$X\in\mathbb{R}^{m\times n}$。

```tex
\documentclass[12pt]{article}
\usepackage{amsfonts}

\begin{document}

$$X\in\mathbb{R}^{m\times n}$$

\end{document}
```

【**例4-29**】使用工具包`bbold`中的`\mathbb`命令书写空心的1、2、3、4、5.

> 参考[The Identity Matrix](https://latex.org/forum/viewtopic.php?f=46&t=20753)

```tex
\documentclass[12pt]{article}
\usepackage{bbold}

\begin{document}

$$\mathbb{1},\mathbb{2},\mathbb{3},\mathbb{4},\mathbb{5}$$

\end{document}
```

编译上述代码，得到幻灯片如图4.2.15所示。

图4.2.15 编译后的幻灯片效果

### 4.2.5 特殊函数

上标很多时候是用来表示变量的幂，例如`x^{2}`表示$x$的平方，由此可以用上标书写出指数函数如$y=x^{2}$等。与指数函数对应的一类常用函数被称为对数函数，是指数函数的反函数，LaTeX提供了一些跟对数函数相关的命令，包括`\log`、`\ln`，在命令`\log`中，我们可以自行设置底数，而命令`\ln`则表示底数为自然数的对数。

【**例4-30**】书写对数函数$y=\log_{2}x$和$y=\ln x$。

```tex
\documentclass[12pt]{article}
\begin{document}

$$y=\log_{2}x$$
$$y=\ln x$$

\end{document}
```

有时候，为了简化数学表达式，我们可能会采用求和或者连乘的写法，在LaTeX中，求和符号对应的命令为`\sum_{下标}^{上标}`，连乘符号对应的命令为`\prod_{下标}^{上标}`。

【**例4-31**】书写求和公式$$\sum_{i=1}^{n}x_{i}$$和连乘公式$$\prod_{i=1}^{n}x_{i}$$

```tex
\documentclass[12pt]{article}
\begin{document}

$$\sum_{i=1}^{n}x_{i}$$
$$\prod_{i=1}^{n}x_{i}$$

\end{document}
```

另外，在初等数学的几何中，我们学过了正弦、余弦等，这些在LaTeX都有定义好的命令可供直接使用。

【**例4-32**】书写正弦函数$y=\sin x$、反正弦函数$y=\arcsin x$、余弦函数$y=\cos x$、反余弦函数$y=\arccos x$、正切函数$y=\tan x$和反正切函数$y=\arctan x$。

```tex
\documentclass[12pt]{article}
\begin{document}

$$y=\sin x$$
$$y=\arcsin x$$
$$y=\cos x$$
$$y=\arccos x$$
$$y=\tan x$$
$$y=\arctan x$$

\end{document}
```

### 练习题

> 打开LaTeX在线系统[https://www.overleaf.com](https://www.overleaf.com/project)或本地安装好的LaTeX编辑器，创建名为LaTeX_practice的项目，并同时新建一个以`.tex`为拓展名的源文件，完成以下几道练习题。

[1] 使用工具包`bbm`中的`\mathbbm`命令书写空心的1和2。

> 参考[Blackboard bold numbers; trouble with \mathbb](https://latex.org/forum/viewtopic.php?f=48&t=17427)

```tex
\documentclass[12pt]{article}
% 请在此处申明使用bbm工具包

\begin{document}

$$\mathbbm{1},\mathbbm{2}$$

\end{document}
```

【回放】[**4.1 基本介绍**](https://nbviewer.jupyter.org/github/xinychen/latex-cookbook/blob/main/chapter-4/section1.ipynb)

【继续】[**4.3 希腊字母**](https://nbviewer.jupyter.org/github/xinychen/latex-cookbook/blob/main/chapter-4/section3.ipynb)

### License

This work is released under the MIT license.


```julia
using MomentClosure, Catalyst, Latexify
```

    ┌ Info: Precompiling MomentClosure [01a1b25a-ecf0-48c5-ae58-55bfd5393600]
    └ @ Base loading.jl:1278

```julia
@parameters t, k_on, k_off, k_p, γ_p, b
@variables p(t), g(t)

vars = [g, p]

ps = [k_on, k_off, k_p, γ_p, b]

S = [1 -1 0 0;
     0 0 b -1]

as = [k_on*(1-g),    # 0 -> g
      k_off*g*(p^2), # g -> 0
      k_p*g,         # 0 -> mP, m ~ Geometric(mean=b)
      γ_p*p]         # p -> 0

binary_vars = [1]

rn = ReactionSystemMod(t, vars, ps, as, S)
```

    ReactionSystemMod(t, Term{Real,Nothing}[g(t), p(t)], Sym{ModelingToolkit.Parameter{Real},Nothing}[k_on, k_off, k_p, γ_p, b], SymbolicUtils.Mul{Real,Int64,Dict{Any,Number},Nothing}[k_on*(1 - g(t)), k_off*g(t)*(p(t)^2), k_p*g(t), γ_p*p(t)], Any[1 -1 0 0; 0 0 b -1])

```julia
raw_eqs = generate_raw_moment_eqs(rn, 3);
```

```julia
latexify(raw_eqs.odes)
```

\begin{align}
\frac{d\mu{_{10}}(t)}{dt} =& k_{on} -1.0 k_{off} \mathrm{\mu{_{12}}}\left( t \right) -1 k_{on} \mathrm{\mu{_{10}}}\left( t \right) \\
\frac{d\mu{_{01}}(t)}{dt} =& b k_{p} \mathrm{\mu{_{10}}}\left( t \right) -1.0 \gamma_{p} \mathrm{\mu{_{01}}}\left( t \right) \\
\frac{d\mu{_{20}}(t)}{dt} =& k_{on} + k_{off} \mathrm{\mu{_{12}}}\left( t \right) + k_{on} \mathrm{\mu{_{10}}}\left( t \right) -2.0 k_{off} \mathrm{\mu{_{22}}}\left( t \right) -2.0 k_{on} \mathrm{\mu{_{20}}}\left( t \right) \\
\frac{d\mu{_{11}}(t)}{dt} =& k_{on} \mathrm{\mu{_{01}}}\left( t \right) + b k_{p} \mathrm{\mu{_{20}}}\left( t \right) -1.0 k_{off} \mathrm{\mu{_{13}}}\left( t \right) -1 k_{on} \mathrm{\mu{_{11}}}\left( t \right) -1.0 \gamma_{p} \mathrm{\mu{_{11}}}\left( t \right) \\
\frac{d\mu{_{02}}(t)}{dt} =& \gamma_{p} \mathrm{\mu{_{01}}}\left( t \right) + b k_{p} \mathrm{\mu{_{10}}}\left( t \right) -2.0 \gamma_{p} \mathrm{\mu{_{02}}}\left( t \right) + 2 b k_{p} \mathrm{\mu{_{11}}}\left( t \right) + 2 k_{p} \mathrm{\mu{_{10}}}\left( t \right) b^{2} \\
\frac{d\mu{_{30}}(t)}{dt} =& k_{on} -1.0 k_{off} \mathrm{\mu{_{12}}}\left( t \right) + 3.0 k_{off} \mathrm{\mu{_{22}}}\left( t \right) -3.0 k_{off} \mathrm{\mu{_{32}}}\left( t \right) + 2.0 k_{on} \mathrm{\mu{_{10}}}\left( t \right) -3.0 k_{on} \mathrm{\mu{_{30}}}\left( t \right) \\
\frac{d\mu{_{21}}(t)}{dt} =& k_{off} \mathrm{\mu{_{13}}}\left( t \right) + k_{on} \mathrm{\mu{_{01}}}\left( t \right) + k_{on} \mathrm{\mu{_{11}}}\left( t \right) + b k_{p} \mathrm{\mu{_{30}}}\left( t \right) -2.0 k_{off} \mathrm{\mu{_{23}}}\left( t \right) -2.0 k_{on} \mathrm{\mu{_{21}}}\left( t \right) -1.0 \gamma_{p} \mathrm{\mu{_{21}}}\left( t \right) \\
\frac{d\mu{_{12}}(t)}{dt} =& k_{on} \mathrm{\mu{_{02}}}\left( t \right) + \gamma_{p} \mathrm{\mu{_{11}}}\left( t \right) + b k_{p} \mathrm{\mu{_{20}}}\left( t \right) -1.0 k_{off} \mathrm{\mu{_{14}}}\left( t \right) -1 k_{on} \mathrm{\mu{_{12}}}\left( t \right) -2.0 \gamma_{p} \mathrm{\mu{_{12}}}\left( t \right) + 2 b k_{p} \mathrm{\mu{_{21}}}\left( t \right) + 2 k_{p} \mathrm{\mu{_{20}}}\left( t \right) b^{2} \\
\frac{d\mu{_{03}}(t)}{dt} =& b k_{p} \mathrm{\mu{_{10}}}\left( t \right) -1.0 \gamma_{p} \mathrm{\mu{_{01}}}\left( t \right) + 3.0 \gamma_{p} \mathrm{\mu{_{02}}}\left( t \right) -3.0 \gamma_{p} \mathrm{\mu{_{03}}}\left( t \right) + 3 b k_{p} \mathrm{\mu{_{11}}}\left( t \right) + 3 b k_{p} \mathrm{\mu{_{12}}}\left( t \right) + 6 k_{p} \mathrm{\mu{_{10}}}\left( t \right) b^{2} + 6 k_{p} \mathrm{\mu{_{10}}}\left( t \right) b^{3} + 6 k_{p} \mathrm{\mu{_{11}}}\left( t \right) b^{2}
\end{align}

```julia
clean_raw_eqs = MomentClosure.bernoulli_moment_eqs(raw_eqs, binary_vars)
latexify(clean_raw_eqs)
```

\begin{align*}
\frac{d\mu{_{10}}}{dt} =& k_{on} - k_{off} \mu{_{12}} - k_{on} \mu{_{10}} \\
\frac{d\mu{_{01}}}{dt} =& b k_{p} \mu{_{10}} - \gamma_{p} \mu{_{01}} \\
\frac{d\mu{_{11}}}{dt} =& k_{on} \mu{_{01}} + b k_{p} \mu{_{10}} - k_{off} \mu{_{13}} - k_{on} \mu{_{11}} - \gamma_{p} \mu{_{11}} \\
\frac{d\mu{_{02}}}{dt} =& \gamma_{p} \mu{_{01}} + b k_{p} \mu{_{10}} + 2 b k_{p} \mu{_{11}} + 2 k_{p} \mu{_{10}} b^{2} - 2 \gamma_{p} \mu{_{02}} \\
\frac{d\mu{_{12}}}{dt} =& k_{on} \mu{_{02}} + \gamma_{p} \mu{_{11}} + b k_{p} \mu{_{10}} + 2 b k_{p} \mu{_{11}} + 2 k_{p} \mu{_{10}} b^{2} - k_{off} \mu{_{14}} - k_{on} \mu{_{12}} - 2 \gamma_{p} \mu{_{12}} \\
\frac{d\mu{_{03}}}{dt} =& b k_{p} \mu{_{10}} + 3 \gamma_{p} \mu{_{02}} + 3 b k_{p} \mu{_{11}} + 3 b k_{p} \mu{_{12}} + 6 k_{p} \mu{_{10}} b^{2} + 6 k_{p} \mu{_{10}} b^{3} + 6 k_{p} \mu{_{11}} b^{2} - \gamma_{p} \mu{_{01}} - 3 \gamma_{p} \mu{_{03}}
\end{align*}

```julia
closed_raw_eqs = moment_closure(raw_eqs, "conditional gaussian", binary_vars);
```

```julia
latexify(closed_raw_eqs)
```

\begin{align*}
\frac{d\mu{_{10}}}{dt} =& k_{on} - k_{off} \mu{_{12}} - k_{on} \mu{_{10}} \\
\frac{d\mu{_{01}}}{dt} =& b k_{p} \mu{_{10}} - \gamma_{p} \mu{_{01}} \\
\frac{d\mu{_{11}}}{dt} =& k_{on} \mu{_{01}} + b k_{p} \mu{_{10}} + 2 k_{off} \mu{_{10}}^{-2} \mu{_{11}}^{3} - k_{on} \mu{_{11}} - \gamma_{p} \mu{_{11}} - 3 k_{off} \mu{_{11}} \mu{_{12}} \mu{_{10}}^{-1} \\
\frac{d\mu{_{02}}}{dt} =& \gamma_{p} \mu{_{01}} + b k_{p} \mu{_{10}} + 2 b k_{p} \mu{_{11}} + 2 k_{p} \mu{_{10}} b^{2} - 2 \gamma_{p} \mu{_{02}} \\
\frac{d\mu{_{12}}}{dt} =& k_{on} \mu{_{02}} + \gamma_{p} \mu{_{11}} + b k_{p} \mu{_{10}} + 2 b k_{p} \mu{_{11}} + 2 k_{off} \mu{_{10}}^{-3} \mu{_{11}}^{4} + 2 k_{p} \mu{_{10}} b^{2} - k_{on} \mu{_{12}} - 2 \gamma_{p} \mu{_{12}} - 3 k_{off} \mu{_{10}}^{-1} \mu{_{12}}^{2} \\
\frac{d\mu{_{03}}}{dt} =& b k_{p} \mu{_{10}} + 3 \gamma_{p} \mu{_{02}} + 3 b k_{p} \mu{_{11}} + 3 b k_{p} \mu{_{12}} + 6 k_{p} \mu{_{10}} b^{2} + 6 k_{p} \mu{_{10}} b^{3} + 6 k_{p} \mu{_{11}} b^{2} - \gamma_{p} \mu{_{01}} - 3 \gamma_{p} \mu{_{03}}
\end{align*}

```julia
latexify(closed_raw_eqs, :closure)
```

\begin{align*}
\mu{_{13}} =& 3 \mu{_{11}} \mu{_{12}} \mu{_{10}}^{-1} - 2 \mu{_{10}}^{-2} \mu{_{11}}^{3} \\
\mu{_{04}} =& 6 \mu{_{01}}^{4} + 3 \mu{_{02}}^{2} + 4 \mu{_{01}} \mu{_{03}} - 12 \mu{_{02}} \mu{_{01}}^{2} \\
\mu{_{14}} =& 3 \mu{_{10}}^{-1} \mu{_{12}}^{2} + 6 \mu{_{10}}^{-3} \mu{_{11}}^{4} + 4 \mu{_{11}} \mu{_{13}} \mu{_{10}}^{-1} - 12 \mu{_{12}} \mu{_{10}}^{-2} \mu{_{11}}^{2} \\
\mu{_{05}} =& 5 \mu{_{01}} \mu{_{04}} + 10 \mu{_{02}} \mu{_{03}} + 60 \mu{_{02}} \mu{_{01}}^{3} - 24 \mu{_{01}}^{5} - 30 \mu{_{01}} \mu{_{02}}^{2} - 20 \mu{_{03}} \mu{_{01}}^{2}
\end{align*}

```julia
# PARAMETER INITIALISATION
mean_p = 200
mean_b = 70
γ_p_val = 1
k_off_val = 0.001
k_on_val = 0.05

k_p_val = mean_p * γ_p_val * (k_off_val * mean_p^2 + k_on_val) / (k_on_val * mean_b)

pmap = [k_on => k_on_val,
        k_off => k_off_val,
        k_p => k_p_val,
        γ_p => γ_p_val,
        b => mean_b]

# DETERMINISTIC INITIAL CONDITIONS
μ₀ = [1, 1]
u₀map = deterministic_IC(μ₀, closed_raw_eqs)

# time interval to solve on
tspan = (0., 6.0)
dt = 0.01

@time oprob = ODEProblem(closed_raw_eqs, u₀map, tspan, pmap);

using OrdinaryDiffEq
@time sol_raw = solve(oprob, Tsit5(), saveat=dt);
```

     16.671893 seconds (17.28 M allocations: 904.953 MiB, 2.99% gc time)
     16.699897 seconds (23.52 M allocations: 1.245 GiB, 20.82% gc time)

```julia
central_eqs = generate_central_moment_eqs(rn, 3);
closed_central_eqs = moment_closure(central_eqs, "conditional gaussian", binary_vars);
latexify(closed_central_eqs, env=:mdtable)
```

|                                                                                $\frac{d\mu{_{10}}}{dt} = k_{on} - k_{off} M{_{12}} - k_{on} \mu{_{10}} - k_{off} M{_{02}} \mu{_{10}} - 2 k_{off} M{_{11}} \mu{_{01}} - k_{off} \mu{_{10}} \mu{_{01}}^{2}$ |
| -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|
|                                                                                $\frac{d\mu{_{01}}}{dt} = b k_{p} \mu{_{10}} - \gamma_{p} \mu{_{01}}$ |
|                                                                                $\frac{dM{_{11}}}{dt} = b k_{p} \mu{_{10}} + 2 k_{off} M{_{11}}^{3} \mu{_{10}}^{-2} - k_{on} M{_{11}} - \gamma_{p} M{_{11}} - b k_{p} \mu{_{10}}^{2} - 3 k_{off} M{_{02}} M{_{11}} - k_{off} M{_{11}} \mu{_{01}}^{2} - 2 k_{off} M{_{12}} \mu{_{01}} - 2 k_{off} M{_{02}} \mu{_{01}} \mu{_{10}} - 3 k_{off} M{_{11}} M{_{12}} \mu{_{10}}^{-1}$ |
|                                                                                $\frac{dM{_{02}}}{dt} = \gamma_{p} \mu{_{01}} + b k_{p} \mu{_{10}} + 2 b k_{p} M{_{11}} + 2 k_{p} \mu{_{10}} b^{2} - 2 \gamma_{p} M{_{02}}$ |
| $\frac{dM{_{12}}}{dt} = \gamma_{p} M{_{11}} + b k_{p} \mu{_{10}} + 2 b k_{p} M{_{11}} + 2 k_{off} M{_{11}}^{4} \mu{_{10}}^{-3} + 2 k_{p} \mu{_{10}} b^{2} + 4 k_{off} \mu{_{01}} M{_{11}}^{3} \mu{_{10}}^{-2} - k_{on} M{_{12}} - 2 \gamma_{p} M{_{12}} - b k_{p} \mu{_{10}}^{2} - 5 k_{off} M{_{02}} M{_{12}} - k_{off} M{_{12}} \mu{_{01}}^{2} - 2 k_{off} \mu{_{10}} M{_{02}}^{2} - 3 k_{off} M{_{12}}^{2} \mu{_{10}}^{-1} - 2 k_{p} b^{2} \mu{_{10}}^{2} - 4 b k_{p} M{_{11}} \mu{_{10}} - 4 k_{off} M{_{02}} M{_{11}} \mu{_{01}} - 6 k_{off} M{_{11}} M{_{12}} \mu{_{01}} \mu{_{10}}^{-1}$ |
|                                                                                $\frac{dM{_{03}}}{dt} = b k_{p} \mu{_{10}} + 3 \gamma_{p} M{_{02}} + 3 b k_{p} M{_{11}} + 3 b k_{p} M{_{12}} + 6 k_{p} M{_{11}} b^{2} + 6 k_{p} \mu{_{10}} b^{2} + 6 k_{p} \mu{_{10}} b^{3} - 3 \gamma_{p} M{_{03}} - \gamma_{p} \mu{_{01}}$ |

```julia
latexify(closed_central_eqs)
```

\begin{align*}
\frac{d\mu{_{10}}}{dt} =& k_{on} - k_{off} M{_{12}} - k_{on} \mu{_{10}} - k_{off} M{_{02}} \mu{_{10}} - 2 k_{off} M{_{11}} \mu{_{01}} - k_{off} \mu{_{10}} \mu{_{01}}^{2} \\
\frac{d\mu{_{01}}}{dt} =& b k_{p} \mu{_{10}} - \gamma_{p} \mu{_{01}} \\
\frac{dM{_{11}}}{dt} =& b k_{p} \mu{_{10}} + 2 k_{off} M{_{11}}^{3} \mu{_{10}}^{-2} - k_{on} M{_{11}} - \gamma_{p} M{_{11}} - b k_{p} \mu{_{10}}^{2} - 3 k_{off} M{_{02}} M{_{11}} - k_{off} M{_{11}} \mu{_{01}}^{2} - 2 k_{off} M{_{12}} \mu{_{01}} - 2 k_{off} M{_{02}} \mu{_{01}} \mu{_{10}} - 3 k_{off} M{_{11}} M{_{12}} \mu{_{10}}^{-1} \\
\frac{dM{_{02}}}{dt} =& \gamma_{p} \mu{_{01}} + b k_{p} \mu{_{10}} + 2 b k_{p} M{_{11}} + 2 k_{p} \mu{_{10}} b^{2} - 2 \gamma_{p} M{_{02}} \\
\frac{dM{_{12}}}{dt} =& \gamma_{p} M{_{11}} + b k_{p} \mu{_{10}} + 2 b k_{p} M{_{11}} + 2 k_{off} M{_{11}}^{4} \mu{_{10}}^{-3} + 2 k_{p} \mu{_{10}} b^{2} + 4 k_{off} \mu{_{01}} M{_{11}}^{3} \mu{_{10}}^{-2} - k_{on} M{_{12}} - 2 \gamma_{p} M{_{12}} - b k_{p} \mu{_{10}}^{2} - 5 k_{off} M{_{02}} M{_{12}} - k_{off} M{_{12}} \mu{_{01}}^{2} - 2 k_{off} \mu{_{10}} M{_{02}}^{2} - 3 k_{off} M{_{12}}^{2} \mu{_{10}}^{-1} - 2 k_{p} b^{2} \mu{_{10}}^{2} - 4 b k_{p} M{_{11}} \mu{_{10}} - 4 k_{off} M{_{02}} M{_{11}} \mu{_{01}} - 6 k_{off} M{_{11}} M{_{12}} \mu{_{01}} \mu{_{10}}^{-1} \\
\frac{dM{_{03}}}{dt} =& b k_{p} \mu{_{10}} + 3 \gamma_{p} M{_{02}} + 3 b k_{p} M{_{11}} + 3 b k_{p} M{_{12}} + 6 k_{p} M{_{11}} b^{2} + 6 k_{p} \mu{_{10}} b^{2} + 6 k_{p} \mu{_{10}} b^{3} - 3 \gamma_{p} M{_{03}} - \gamma_{p} \mu{_{01}}
\end{align*}

```julia
u₀map = deterministic_IC(μ₀, closed_central_eqs)

@time oprob = ODEProblem(closed_central_eqs, u₀map, tspan, pmap);
@time sol_central = solve(oprob, Tsit5(), saveat=dt);
```

      1.855873 seconds (2.18 M allocations: 112.563 MiB, 1.92% gc time)
      2.297089 seconds (3.13 M allocations: 146.251 MiB, 1.51% gc time)

```julia
using Plots

plot(sol_central.t, [sol_raw[1,:], sol_central[1,:]],
    label  = ["Raw" "Central"],
    legend = true,
    xlabel = "Time [hr]",
    ylabel = "Mean number of proteins",
    lw=2,
    legendfontsize=8,
    xtickfontsize=10,
    ytickfontsize=10,
    dpi=100)
```

```julia
plot(sol_central.t, [sol_raw[2,:], sol_central[2,:]],
    label  = ["Raw" "Central"],
    legend = true,
    xlabel = "Time [hr]",
    ylabel = "Mean number of proteins",
    lw=2,
    legendfontsize=8,
    xtickfontsize=10,
    ytickfontsize=10,
    dpi=100)
```

```julia
raw_std = sqrt.(sol_raw[4,2:end] .- sol_raw[2,2:end].^2)
central_std = sqrt.(sol_central[4,2:end])
plot(sol_central.t[2:end], [raw_std, central_std],
    label  = ["Raw" "Central"],
    legend = true,
    xlabel = "Time [hr]",
    ylabel = "Mean number of proteins",
    lw=2,
    legendfontsize=8,
    xtickfontsize=10,
    ytickfontsize=10,
    dpi=100)
```


# Examples for Voigt profiles (v1.2)

```python
%matplotlib inline
```

```python
### import
import astropy.units as u
from linetools.spectralline import AbsLine
from linetools.spectra import io as lsio
from linetools.analysis import voigt as lav
# plots
from matplotlib import pyplot as plt
import pylab
pylab.rcParams['figure.figsize'] = (8.0, 6.0)
import matplotlib
matplotlib.rc('xtick', labelsize=20)
matplotlib.rc('ytick', labelsize=20)
```

```python
# Define a plotting function
def plt_line(spec):
    plt.clf()
    plt.figure(dpi=1200)
    plt.plot(spec.dispersion.value, spec.flux.value, 'k-', drawstyle='steps-mid', lw=1.5)
    plt.xlim(3644., 3650.)
    plt.ylabel('Normalized Flux', fontsize=20.)
    plt.xlabel('Wavelength', fontsize=20.)
    ax = plt.gca()
    ax.xaxis.set_major_locator(plt.MultipleLocator(2.))
    ax.get_xaxis().get_major_formatter().set_useOffset(False)
    plt.ylim(0., 1.1)
    plt.show()
    plt.close()
```

## Generate a Ly$\alpha$ line

```python
abslin = AbsLine(1215.670*u.AA)
```

    WARNING: UnitsWarning: The unit 'Angstrom' has been deprecated in the FITS standard. Suggested: 10**-1 nm. [astropy.units.format.utils]
    WARNING:astropy:UnitsWarning: The unit 'Angstrom' has been deprecated in the FITS standard. Suggested: 10**-1 nm.

## Fill attributes

```python
abslin.attrib['N'] = 10**14./u.cm**2  # log N
abslin.attrib['b'] = 25.*u.km/u.s
abslin.attrib['z'] = 2.0
```

## Generate Voigt profile

### Use wavelength from spectrum

```python
#Load
abslin.analy['spec'] = lsio.readspec('../../linetools/spectra/tests/files/UM184_nF.fits')
```

```python
#Generate
vmodel = abslin.generate_voigt()
```

    /Users/ncrighton/Code/Repo/linetools/build/lib.macosx-10.5-x86_64-3.4/linetools/analysis/voigt.py:181: UserWarning: Using a sub-grid wavelength array because the input array is too coarse.
      warnings.warn('Using a sub-grid wavelength array because the input array is too coarse.')
    /Users/ncrighton/Code/Repo/linetools/build/lib.macosx-10.5-x86_64-3.4/linetools/analysis/voigt.py:182: UserWarning: Will return values rebinned to the input array.
      warnings.warn('Will return values rebinned to the input array.')
    /Users/ncrighton/Code/Repo/linetools/build/lib.macosx-10.5-x86_64-3.4/linetools/analysis/voigt.py:235: UserWarning: Assuming infinite spectral resolution, i.e. no smoothing.
      warnings.warn('Assuming infinite spectral resolution, i.e. no smoothing.')
    /Users/ncrighton/Code/Repo/linetools/build/lib.macosx-10.5-x86_64-3.4/linetools/analysis/voigt.py:236: UserWarning: Set fwhm to smooth.
      warnings.warn('Set fwhm to smooth.')

```python
# Plot
plt_line(vmodel)
```

### Use self-generated wavelength

```python
abslin.attrib['N'] = 10**17.5/u.cm**2
abslin.attrib['b'] = 20.*u.km/u.s
```

```python
wave = np.linspace(3644, 3650,100)*u.AA
```

```python
vmodel2 = abslin.generate_voigt(wave=wave)
```

    /Users/ncrighton/Code/Repo/linetools/build/lib.macosx-10.5-x86_64-3.4/linetools/analysis/voigt.py:181: UserWarning: Using a sub-grid wavelength array because the input array is too coarse.
      warnings.warn('Using a sub-grid wavelength array because the input array is too coarse.')
    /Users/ncrighton/Code/Repo/linetools/build/lib.macosx-10.5-x86_64-3.4/linetools/analysis/voigt.py:182: UserWarning: Will return values rebinned to the input array.
      warnings.warn('Will return values rebinned to the input array.')
    /Users/ncrighton/Code/Repo/linetools/build/lib.macosx-10.5-x86_64-3.4/linetools/analysis/voigt.py:235: UserWarning: Assuming infinite spectral resolution, i.e. no smoothing.
      warnings.warn('Assuming infinite spectral resolution, i.e. no smoothing.')
    /Users/ncrighton/Code/Repo/linetools/build/lib.macosx-10.5-x86_64-3.4/linetools/analysis/voigt.py:236: UserWarning: Set fwhm to smooth.
      warnings.warn('Set fwhm to smooth.')

```python
plt_line(vmodel2)
```

## Multiple Lines

```python
abslin2 = AbsLine('DI 1215')
abslin2.attrib['N'] = 10**13./u.cm**2  # log N
abslin2.attrib['b'] = 15.*u.km/u.s
abslin2.attrib['z'] = 2.0
```

```python
vmodel3 = lav.voigt_from_abslines(wave,[abslin,abslin2])
```

    /Users/ncrighton/Code/Repo/linetools/build/lib.macosx-10.5-x86_64-3.4/linetools/analysis/voigt.py:181: UserWarning: Using a sub-grid wavelength array because the input array is too coarse.
      warnings.warn('Using a sub-grid wavelength array because the input array is too coarse.')
    /Users/ncrighton/Code/Repo/linetools/build/lib.macosx-10.5-x86_64-3.4/linetools/analysis/voigt.py:182: UserWarning: Will return values rebinned to the input array.
      warnings.warn('Will return values rebinned to the input array.')
    /Users/ncrighton/Code/Repo/linetools/build/lib.macosx-10.5-x86_64-3.4/linetools/analysis/voigt.py:235: UserWarning: Assuming infinite spectral resolution, i.e. no smoothing.
      warnings.warn('Assuming infinite spectral resolution, i.e. no smoothing.')
    /Users/ncrighton/Code/Repo/linetools/build/lib.macosx-10.5-x86_64-3.4/linetools/analysis/voigt.py:236: UserWarning: Set fwhm to smooth.
      warnings.warn('Set fwhm to smooth.')

```python
plt_line(vmodel3)
```

## $\tau$ Profiles

```python
tau = lav.voigt_from_abslines(wave,abslin,ret='tau')
```

    /Users/ncrighton/Code/Repo/linetools/build/lib.macosx-10.5-x86_64-3.4/linetools/analysis/voigt.py:181: UserWarning: Using a sub-grid wavelength array because the input array is too coarse.
      warnings.warn('Using a sub-grid wavelength array because the input array is too coarse.')
    /Users/ncrighton/Code/Repo/linetools/build/lib.macosx-10.5-x86_64-3.4/linetools/analysis/voigt.py:182: UserWarning: Will return values rebinned to the input array.
      warnings.warn('Will return values rebinned to the input array.')
    /Users/ncrighton/Code/Repo/linetools/build/lib.macosx-10.5-x86_64-3.4/linetools/analysis/voigt.py:220: UserWarning: Rebinned tau back to your input array.  Reconsider input
      warnings.warn('Rebinned tau back to your input array.  Reconsider input')

```python
#
plt.clf()
plt.figure(dpi=1200)
plt.plot(wave,tau, 'k-', drawstyle='steps-mid', lw=1.5)
plt.xlim(3644., 3650.)
plt.ylabel('Optical Depth', fontsize=20.)
plt.xlabel('Wavelength', fontsize=20.)
plt.show()
plt.close()
```

## Fit the Voigt

```python
from astropy.modeling import fitting
```

```python
fitvoigt = lav.single_voigt_model(logN=np.log10(abslin.attrib['N'].value),b=abslin.attrib['b'].value,
                                z=abslin.attrib['z'], wrest=abslin.wrest.value,
                                gamma=abslin.data['gamma'].value, f=abslin.data['f'],
                                 fwhm=3.)
```

```python
fitter = fitting.LevMarLSQFitter()
```

```python
p = fitter(fitvoigt,wave.value,vmodel2.flux.value,weights=1./(np.ones(len(wave))*0.1))
print(p)
```

    Model: single_voigt_model
    Inputs: ('wave',)
    Outputs: ('flux',)
    Model set size: 1
    Parameters:
             logN           b            z        wrest    f       gamma    fwhm
        ------------- ------------- ------------ ------- ------ ----------- ----
        17.4929562589 20.0104211026 1.9999973925 1215.67 0.4164 626500000.0  3.0

```python
#
plt.clf()
plt.plot(wave, vmodel2.flux, 'k-',drawstyle='steps')
plt.plot(wave,p(wave.value), 'g-')
#xdb.xplot(wave,model.flux,p(wave.value))
plt.show()
plt.close()
# note the bad pixels at either end of the spectrum - this is a bug! It can't be
# worked around for now by making sure that the spectrum is much larger than the region
# of interest, and discarding the first and last ~10 pixels.
```

## Test Voigt Models for Speed and Accuracy

```python
import timeit
```

### Benchmark for accuracy

```python
def voigt_slow(a, u):
    """ Calculate the voigt function to very high accuracy.
    Uses numerical integration, so is slow.  Answer is correct to 20
    significant figures.
    Note this needs `mpmath` or `sympy` to be installed.
    """
    try:
        import mpmath as mp
    except ImportError:
        from sympy import mpmath as mp
    with mp.workdps(20):
        z = mp.mpc(u, a)
        result = mp.exp(-z*z) * mp.erfc(-1j*z)

    return result.real
```

```python
# This cell takes a few minutes to run.

a = 0.05
alluvoigt = np.linspace(0.,5.,1000)
true_voigt05 = []
%timeit -r 10 for uvoigt in alluvoigt: true_voigt05.append(voigt_slow(a,uvoigt))
# Just take first 1000
true_voigt05 = np.array(true_voigt05[0:1000])
```

    1 loops, best of 10: 53.8 s per loop

```python
# Plot
plt.clf()
plt.plot(alluvoigt, true_voigt05, '-')
plt.show()
plt.close()
```

### VoigtKing

#### Speed

```python
%timeit -r 10 king_voigt05 = lav.voigtking(alluvoigt,a)
```

#### Accuracy

```python
king_voigt05 = lav.voigtking(alluvoigt,a)
plt.clf()
plt.plot(alluvoigt, king_voigt05-true_voigt05, 'k-')
plt.show()
plt.close()
```

### Scipy

```python
from scipy.special import wofz
def voigt_wofz(u,a):
    return wofz(u + 1j * a).real
```

#### Speed

```python
%timeit -r 10 wofz_voigt05 = voigt_wofz(alluvoigt,a)
```

    10000 loops, best of 10: 129 µs per loop

#### Accuracy

```python
wofz_voigt05 = voigt_wofz(alluvoigt,a)
#
plt.clf()
plt.plot(alluvoigt, wofz_voigt05-true_voigt05, 'g-')
plt.show()
plt.close()
```


# **Universal Qubit Classifier Using Quantum Gradient Descent**

### This Notebook is created with love by [Alexandria Quantum Computing Group](http://www.sci.p.alexu.edu.eg/~aleqcg/)

###### *Author: Kareem El-Safty* [mail](kareemessafty@gmail.com)

###### This Notebook's purpose is to introduce machine learning developers or researchers to the Quantum Machine Learning (QML) field. We will implement the following [paper](https://arxiv.org/abs/1907.02085) to show the potential of quantum machine learning. The [original repo](https://github.com/AdrianPerezSalinas/universal_qlassifier) of the paper

### Important Disclaimer
This notebook is inspired from this [tutorial](https://pennylane.ai/qml/app/tutorial_data_reuploading_classifier.html) except for some changes I have made.

## **Introduction**

First of all, there is no need to have a solid background in quantum physics to go through this notebook. It would, of course, be helpful but little experience is assumed. However, a solid knowledge of **algebra**, some **calculus**, **trigonometry**, **programming**, and some **machine learning experience** are extremly important.

Throughout the notebook, there will be some reference links that describe some notions or show the origins of a specific assumption. You are advised to follow them if by any chance you feel little uncomfortable with what you read. It's safe to say that in this notebook you may find most of the answers that you are looking for or at least a doorway for them.

### Environement Setup:
Google Colab does provide everything we need in this tutorial but if you want to try this on your machine then a working version of Python 3.5 or higher would be great. Running the following command line will setup everything for us

> pip install qiskit --upgrade

Some linux machines may require a sudo previlage, so you should pay some intension to that. To make sure everything is working perfect check the output of the following code in a jupyter notebook:

```
import qiskit.tools.jupyter
%qiskit_version_table
%qiskit_copyright
```
It should print out the current qiskit version which is 0.11.1 and also the other accompanying libraries. If you want to print out the version in a normal ipython terminal then copy the follwing code:

```
import qiskit
print(qiskit.__version__)
```

Let's first execute the first cell to install the necessary packages on colab.

```python
!pip install qiskit -q --upgrade

import qiskit.tools.jupyter
%qiskit_version_table
%qiskit_copyright
```

Version Information

Qiskit Software	Version

Qiskit	None

Terra	0.11.1

Aer	0.3.4

Ignis	0.2.0

Aqua	0.6.2

IBM Q Provider	0.4.5

System information

Python	3.6.9 (default, Nov  7 2019, 10:44:02)
[GCC 8.3.0]

OS	Linux

CPUs	1

Memory (Gb)	12.715873718261719

Fri Jan 17 05:57:31 2020 UTC

This code is a part of Qiskit

© Copyright IBM 2017, 2020.

This code is licensed under the Apache License, Version 2.0. You may
obtain a copy of this license in the LICENSE.txt file in the root directory
 of this source tree or at http://www.apache.org/licenses/LICENSE-2.0.
Any modifications or derivative works of this code must retain this
copyright notice, and modified files need to carry a notice indicating
that they have been altered from the originals.

By now, you may be wondering about [Qiskit](https://qiskit.org/). Well, it may appear to you as a normal python package that presents some tools for quantum programming. You're not entirely wrong about that. However, it's IBM's main framework for quantum computing. This means that it gives you the required means to control actual quantum computers, calibrate the errors that may occur during executing any circuit, build different quantum algorithms, and of course visualise most of them. It's safe to say that It's a full stack framework where both Hardware and Software quantum developers are working together. You may find all the details you want on their online website.

## **Problem Definition**

The probelm is simply a binary classification task but with a non-linear sepearable dataset. This dataset is chosen to show that the quantum algorithm used for this tutorial is good enough to tackle similar problems easily. The following cell imports the required libraries to plot the dataset.

```python
from sklearn.datasets import make_moons
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import MinMaxScaler
import matplotlib.pylab as plt

n_samples = 900
test_size = 0.1

data,labels = make_moons(n_samples=n_samples,random_state=5,noise=0.04)

#splitting the data evenly
x_train,x_test,y_train,y_test = train_test_split(data,labels,stratify=labels,test_size=test_size,random_state=0)

#preprocessing
scale = MinMaxScaler(feature_range=(-1,1)).fit(x_train)
x_train = scale.transform(x_train)
x_test = scale.transform(x_test)

print('shape of the training & testing data: ', x_train.shape, x_test.shape)

#simple plotting
plt.scatter(x_train[:,0][y_train==0], x_train[:,1][y_train==0], marker='o',
            color='red', label='class 0')
plt.scatter(x_train[:,0][y_train==1], x_train[:,1][y_train==1], marker='*',
            color='blue', label='class 1')
plt.scatter(x_test[:,0][y_test==0], x_test[:,1][y_test==0], marker='+',
            color='yellow', label='test 0')
plt.scatter(x_test[:,0][y_test==1], x_test[:,1][y_test==1], marker='x',
            color='black', label='test 1')

plt.legend()
plt.show()

print("number of samples for testing class_1: ",(y_test==1).sum())
print("number of samples for testing class_0: ",(y_test==0).sum())
```

The dataset consists of *810* samples for training and *90* samples for testing. They are distributed equally and also scaled between [-1,1]. The typical way to handle such dataset is to use the [Support Vector Machine](http://web.mit.edu/6.034/wwwbob/svm-notes-long-08.pdf), [SVM with scikit-learn](https://scikit-learn.org/stable/modules/svm.html), and [SVM with QISKIT](https://iwatobipen.wordpress.com/2019/07/07/svm-on-quantum-computing-quantumcomputing-qiskit/) .

I do urge you to check the first two links if you are not familiar with it. The third link is a direct implementation of quantum SVM algorithm using qiskit-aqua. You may want to check it after finishing this tutorial or if you already familiar with both quantum computing and machine learning then go ahead and open it.

The following code with show you how the svm can solve this problem easily:

```python
from sklearn.svm import SVC
import numpy as np

model = SVC(kernel = 'rbf', gamma = 'scale', C = 1, random_state = 0)

model.fit(x_train,y_train)

print("accuracy of testing is: ", model.score(x_test,y_test))
```

    accuracy of testing is:  1.0

**plotting the decision boundary**

```python
#creating the surface
xx, yy = np.meshgrid(np.linspace(-1, 1, 500), np.linspace(-1, 1, 500))
X_grid = [np.array([x, y]) for x, y in zip(xx.flatten(), yy.flatten())]
predictions_grid = model.predict(X_grid)
Z = np.reshape(np.array(predictions_grid), xx.shape)

cm = plt.cm.RdBu

cnt = plt.contourf(xx, yy, Z, levels=np.arange(0., 1.1, 0.1), cmap=cm, alpha=.8)

plt.colorbar(cnt, ticks=[0, 0.5, 1])

plt.scatter(x_train[:, 0][y_train==0], x_train[:, 1][y_train==0], c='g',
            marker='^', edgecolors='k',label='train 0')
plt.scatter(x_train[:, 0][y_train==1], x_train[:, 1][y_train==1], c='b',
            marker='*', edgecolors='k',label='train 1')
plt.scatter(x_test[:, 0][y_test==0], x_test[:, 1][y_test==0], c='r',
            marker='+', edgecolors='k',label='test 0')
plt.scatter(x_test[:, 0][y_test==1], x_test[:, 1][y_test==1], c='yellow',
            marker='x', edgecolors='k',label='test 1')

plt.ylim(-1.07, 1.07)
plt.xlim(-1.07, 1.07)

plt.legend()
plt.tight_layout()
plt.show()
print(model.n_support_)
```

 **Voila!** you've done it. Then why bother doing the same thing but in a quantum way. You may find some answers to this question but most of them wouldn't be encourging from a business point of view because you already do have an efficient, fast, and clean way. On the other hand, considering real world scenarios, Neural Networks play the most important and vital role in both academia and industry.

The most reasonable answer for this question is a small history lesson. 15 years ago this [paper](https://lear.inrialpes.fr/people/triggs/pubs/Dalal-cvpr05.pdf) was the true state of the art, the hype and the coolest thing to use for detecting objects and of course recognising them. It used only a ***Linear Kernel*** for the SVM. After 4 to 5 years GPUs started to be affordable to everyone and this simply led to using them for very complex algebric computations.

As a natural consequence this [paper](https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf) by Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton has officially started the deep learning era. However in 1998 this [paper](http://yann.lecun.com/exdb/publis/pdf/lecun-01a.pdf) by Yann LeCun, Leon Bottou, Yoshua Bengio, and Patrick Haffner, has actually presented a very promising convolutional neural network for recognising handwritten english numbers aka the mnist dataset. But it was incredibly hard to use this wonderful and novel architecture with the available hardware back then.

In the next 15 to 20 years from now, Quantum Computers would be more powerful, immune to noise, and most importantly essential for us to move on to the next step of solving classically intractable problems as shown in the following figure:

It's taken from this great [course form MIT](https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-845-quantum-complexity-theory-fall-2010/) about Complexity Theory taughy by Prof. Scott Aaronson.

Quantum Computation is the natural evolution of the information processing industry and whether you agree or disagree with that it's happening. Let's start introducing some important concepts regarding quantum computing.

## **Quantum Computing Basics**

Information like numbers, voice recordings, or text is encoded into simply *ones* and *zeros*. By combining billions of transistors you can build up different architectures of computers to represent whatever you like.

From a quantum point of view instead of a "big transistor" that can only produce an on or an off state at a time, a quantum particle like for example an electron can encode 2 classical states the zero and the one.

Let us clarify this mathematically: [bra-ket notation](https://en.wikipedia.org/wiki/Bra%E2%80%93ket_notation)

\begin{align}
  & \left| \psi  \right\rangle =\alpha \left| 0 \right\rangle +\beta \left| 1 \right\rangle \, $ eq(1)\\ &
  where\,\,  alpha\, \,  and\, \, beta \in \mathbb{C} \\ & {{\alpha }^{2}}+{{\beta }^{2}}=1, \\
 & \operatorname{and}\,\left| \, \right\rangle \,,\left\langle  \, \right|\,\text{are}\,\text{called}\,\text{the}\,\text{ket}\,\text{and}\,\text{bra}\,\text{notations} \\
\end{align}

The quantum state $\psi$ can be represented by its ground and excited states the zero and the one at the same time. $\alpha$ and $\beta$ are complex numbers and determine the possiblity of each eigen vector. So, the zero and the one are not simply numbers but rather they represent the eigen vectors of the system as follows:
\begin{align}
\left| 0 \right\rangle =\left[ \begin{matrix}
   1  \\
   0  \\
\end{matrix} \right]
\end{align}

\begin{align}
\left| 1 \right\rangle =\left[ \begin{matrix}
   0  \\
   1  \\
\end{matrix}\right]
\end{align}

eq(1) can be written in a more general form as follows:
\begin{align}
\left| \psi  \right\rangle \,=\,\cos \left( \frac{\theta }{2} \right)\left| 0 \right\rangle \,+\,{{e}^{i\varphi }}\sin \left( \frac{\theta }{2} \right)\left| 1 \right\rangle \ \ \ \ \ \  \ \ \  \ \ \ \  \ \ \ \ \ \ \  \ \ \ \ \ \ \ \ \  eq (2) \end{align}

Eq(2) resembles a very famous notation which is the [Bloch Sphere](http://www.vcpc.univie.ac.at/~ian/hotlist/qc/talks/bloch-sphere.pdf), [Bloch Sphere cont](http://www.vcpc.univie.ac.at/~ian/hotlist/qc/talks/bloch-sphere-rotations.pdf). I implore you to check these two links as they build up the first two equations and then explain how can we create any quantum state we want from basic roations as follows:

\begin{align}
  & {{R}_{{\hat{n}}}}\left( \theta  \right)\,=\,\cos \left( \frac{\theta }{2} \right)I\,-\,i\,\sin \left( \frac{\theta }{2} \right)\hat{n}\centerdot \sigma  &         eq(3)\\ \\
 & where\,\,\hat{n}\centerdot \sigma \,=\,\left( {{{\hat{n}}}_{x}}X+\,{{{\hat{n}}}_{y}}Y\,+\,{{{\hat{n}}}_{z}}Z \right) \\ \\ \\
\end{align}

\\begin{align}
  & {{R}_{x}}\left( \theta  \right)={{e}^{-i\frac{\theta X}{2}}}=\cos \left( \frac{\theta }{2} \right)I-i\sin \left( \frac{\theta }{2} \right)X=\left[ \begin{matrix}
   \cos \left( \frac{\theta }{2} \right) & -i\sin \left( \frac{\theta }{2} \right)  \\
   -i\sin \left( \frac{\theta }{2} \right) & \cos \left( \frac{\theta }{2} \right)  \\
\end{matrix} \right]\, \\
 & {{R}_{y}}\left( \theta  \right)={{e}^{-i\frac{\theta \text{Y}}{2}}}=\cos \left( \frac{\theta }{2} \right)I-i\sin \left( \frac{\theta }{2} \right)Y=\left[ \begin{matrix}
   \cos \left( \frac{\theta }{2} \right) & -\sin \left( \frac{\theta }{2} \right)  \\
   \sin \left( \frac{\theta }{2} \right) & \cos \left( \frac{\theta }{2} \right)  \\
\end{matrix} \right] & eq(4) \\
 & {{R}_{z}}\left( \theta  \right)={{e}^{-i\frac{\theta Z}{2}}}=\cos \left( \frac{\theta }{2} \right)I-i\sin \left( \frac{\theta }{2} \right)Z=\left[ \begin{matrix}
   {{e}^{-i\frac{\theta }{2}}} & 0  \\
   0 & {{e}^{i\frac{\theta }{2}}}  \\
\end{matrix} \right] \\ \\
\end{align}

\begin{align}
  & X\,=\,\left[ \begin{matrix}
   0 & 1  \\
   1 & 0  \\
\end{matrix} \right] \\
 & Y=\,\left[ \begin{matrix}
   0 & -i  \\
   i & 0  \\
\end{matrix} \right] & \,\,eq(5)\\
 & Z=\left[ \begin{matrix}
   1 & 0  \\
   0 & -1  \\
\end{matrix} \right] \\
 & I\,=\,\left[ \begin{matrix}
   1 & 0  \\
   0 & 1  \\
\end{matrix} \right] \\ \\\\
\end{align}

\begin{align}
  & X\left| 0 \right\rangle \,=\,\left| 1 \right\rangle ,\,\,\,\,X\left| 1 \right\rangle =\left| 0 \right\rangle  \\
 & Y\left| 0 \right\rangle \,=\,i\left| 1 \right\rangle ,\,\,\,Y\left| 1 \right\rangle \,=\,-i\left| 0 \right\rangle  & eq(6)\\
 & Z\left| 0 \right\rangle \,=\,\left| 0 \right\rangle ,\,\,\,Z\left| 1 \right\rangle \,=-\left| 1 \right\rangle  \\ \\
\end{align}

Let us break these down one by one:

*   The implicaitons of the first equation are that any quantum state can be in a superposition of two different states, i.e. it might be in 0, or 1 at the same exact time.

*   $\alpha$ and $\beta$ are complex numbers i.e the space where the quantum state exists is a complex hilbert space and also have length of 1 since the sum of the squares equal 1.

*   Since the length of the quantum state is one we can use discard the $r$ component in the spherical cordinates and focus more on the $\theta$ and $\alpha$ components.

*   This will make it easier to find any arbitrary axis and rotate our quantum state inside the bloch shpere as shown in equation $3$

*   If we set $\theta$ to $\pi/2$ equation $5$ will be a special case of $4$ where we defined general rotational Gates that can span the three main axes $X$, $Y$, and $Z$.

*   The following [tool](https://javafxpert.github.io/grok-bloch/?fbclid=IwAR1yC9NJ8ZooQh0zlwGAeChR4l9Xc-WLAvVJf7OhFRMBk8yrX0UoFXvLHOg) will give you a very good intution about what is going on if the maths is annoying.

*   Equation $6$ shows the effet of the basic quantum gates on the eigen vector zero and eigen vector one.

These rotational gates are very important as they enable us to encode our classical data into the quantum states using the free parameters like $\theta$ for example.

There is an important gate called the Rotational gate or $U_3$ where we have three parameters that allow us to encode our classical features in a single qubit.

Let's say we have a vector $x$ that consists of $3$ features like this $x=[x_1,x_2,x_3]$ then $U_3=(\theta,\alpha,\omega)$ will encompase the three features inside each parameter.

## Implementing the quantum circuit

The following cell contains a small snipt of code that shows the idea behind the universal qubit classifier. We start by pading the feature vector with zero and then encode it into the $U_3$ gate. we do the same thing for the second qubit. after that we follow it by another rotational gate but its parameters are randomly initialised. these paramterised gates will be optimized later on.

there is somehting called the controlled $Z$ gate this is a source of entanglement you can check it in this amazing [book](https://qiskit.org/textbook/ch-gates/quantum-gates.html) by Qiskit.

this entanglement is not necessary for the sucess of the architecture but i have included it here for the sake of completeness. the controlled $Z$ is repeated $l-1$ where $l$ is the number of repeated layers.

If the feature vector is bigger than three let's say 6 features then 2 $U_3$ gates will be used to encode the feature and then the same thing would happen again.

```python
from qiskit import QuantumCircuit, QuantumRegister, ClassicalRegister
from qiskit import *

np.random.seed(0)

weights = np.random.rand(2,2,3) # 2 qubits, 2 layers, 3 weights in each layer

qc = QuantumCircuit(2)

qc.u3(x_train[0][0],x_train[0][1],0,0)
qc.u3(x_train[0][0],x_train[0][1],0,1)

qc.u3(weights[0][0][0],weights[0][0][1],weights[0][0][2],0)
qc.u3(weights[1][0][0],weights[1][0][1],weights[1][0][2],1)

qc.cz(0,1)

qc.u3(x_train[0][0],x_train[0][1],0,0)
qc.u3(x_train[0][0],x_train[0][1],0,1)

qc.u3(weights[0][1][0],weights[0][1][1],weights[0][1][2],0)
qc.u3(weights[1][1][0],weights[1][1][1],weights[1][1][2],1)

backend_sim = Aer.get_backend('statevector_simulator')

job_sim = execute(qc, backend_sim)

result_sim = job_sim.result()

state_vector = result_sim.get_statevector(qc)
print(state_vector)
qc.draw(output='mpl',scale=0.8)
```

Since the output from the quantum circuit is simply a vecctor, we can actually assign for each label a vector and then we measure the similarity between them.

As we only have two labels $0$, and $1$ it would be great to encode these labels in the Zero state and One state as mentioned above and then do somehting called [fidelity measurement](https://en.wikipedia.org/wiki/Fidelity_of_quantum_states) where we compute the overlap between the state vector and the other two vectors.
This can be achieved by $|<1|\psi>|^2$ and $|<0|\psi>|^2$

```python
from qiskit.quantum_info import state_fidelity

print('the label of the first sample is: ', y_train[0])

one = np.array([0,0,0,1],dtype='complex')

zero = np.array([1,0,0,0],dtype='complex')

print(state_fidelity(state_vector,one))
print(state_fidelity(state_vector,zero))
```

    the label of the first sample is:  0
    0.07052486680732599
    0.7187336048871401

The higher the fidelity the better the accuracy and as we can see the label of the first sample is $0$ and its fidelity is the higher.

The output from the quantum circuit is then passed into a log loss function to compute the cost fucntion. this is the only difference between the paper and this tutorial where they defined their cost function to be $\frac{1}{m}\times(1-fidelity_m)$ where $m$ is number of samples

***The authors of the paper discusses the possibilities of this architecture and its similarities to the classical neural networks in the second section of the paper and proved its universality as a quantum classifier.***

## Code Implementation

This section will contain all the necessary code to implement the paper using qiskit and it.

The code is flexible to implement as many layers as you want with one or two qubits only you are of course welcome to upgrade the code for more qubits as you want.

The fucntion create_data() must be modified if you want increase the number of features or increase the number of labels to more than $3$

```python
from qiskit import QuantumCircuit, QuantumRegister, ClassicalRegister
from qiskit import *
from qiskit.quantum_info.analysis import average_data
from qiskit.quantum_info import state_fidelity
from sklearn.metrics import accuracy_score,log_loss,confusion_matrix,classification_report
import numpy as np
from tqdm import tqdm
from sklearn.datasets import make_moons
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import MinMaxScaler
import matplotlib.pylab as plt
```

```python
n_qubits = 1
layers = 6
n_samples = 900
```

```python
def cerate_data(n_qubits,layers,dataset_name,n_samples,test_size = 0.1,plot=True,normalize=True):

  """
  this fucntion takes 4 main inputs:
  n_qubits: int 1,....,n
  layers: int 1,....,n
  dataset_name: str like 'moons'
  n_samples: int 2,......,n
  test_size: float between 0 and 1

  it returns the training and the testing dataset with their labels and also the
  corrosponding target states and also the weights for training the classifier
  """

  # creating circles dataset taken from: https://github.com/AdrianPerezSalinas/universal_qlassifier/blob/master/data_gen.py

  def _circle(n_samples):
    centers = np.array([[0, 0]])
    radii = np.array([np.sqrt(1.8/np.pi)])
    data=[]
    dim = 2
    labels=[]
    np.random.seed(1000)
    for i in range(samples):
        x = 2 * (np.random.rand(dim)) - 1
        y = 0
        for c, r in zip(centers, radii):
            if np.linalg.norm(x - c) < r:
                y = 1

        data.append(x)
        labels.append(y)

    return np.array(data), np.array(labels)

  if dataset_name.lower() == 'moons':
    data,labels = make_moons(n_samples=n_samples,random_state=5,noise=0.04)

  if dataset_name.lower() == 'circles':
    data,labels = _circle(n_samples)

  x_train,x_test,y_train,y_test = train_test_split(data,labels,stratify=labels,test_size=test_size,random_state=0) # splitting data evenly

  if normalize: # scale features  [-1:1]

    if len(x_train.shape) == 1: #check for dimension to prevent any error from scaling
      x_train = x_train.reshape(-1,1)
      x_test = x_test.reshape(-1,1)

    scale = MinMaxScaler(feature_range=(-1,1)).fit(x_train)
    x_train = scale.transform(x_train)
    x_test = scale.transform(x_test)

  #check features' dimensions and make it 3 or its multiples
  if len(x_train)<3 or len(x_train) > 6:

    if x_train.shape[1] == 2:

      x_train = np.hstack((x_train,np.zeros(len(x_train)).reshape(-1,1)))
      x_test = np.hstack((x_test,np.zeros(len(x_test)).reshape(-1,1)))

    if x_train.shape[1] == 1:

      x_train = np.hstack((x_train,np.zeros(len(x_train)).reshape(-1,1),np.zeros(len(x_train)).reshape(-1,1)))
      x_test = np.hstack((x_test,np.zeros(len(x_test)).reshape(-1,1),np.zeros(len(x_test)).reshape(-1,1)))

    if x_train.shape[1] == 4:

      x_train = np.hstack((x_train,np.zeros(len(x_train)).reshape(-1,1),np.zeros(len(x_train)).reshape(-1,1)))
      x_test = np.hstack((x_test,np.zeros(len(x_test)).reshape(-1,1),np.zeros(len(x_test)).reshape(-1,1)))

    if x_train.shape[1] == 5:

      x_train = np.hstack((x_train,np.zeros(len(x_train)).reshape(-1,1)))
      x_test = np.hstack((x_test,np.zeros(len(x_test)).reshape(-1,1)))
    if x_train.shape[1] > 6:
      raise ValueError("please create a dataset whose number of features are divisible by 3 as there is no checking mechanisim implemented for values bigger than 6")

  # prepare target states based on number of labels, currently only two-label is implemented

  n_of_labels = len(np.unique(labels))

  state_labels = np.zeros((n_of_labels,2**n_qubits),dtype=np.complex128)

  if n_of_labels == 2:
    state_labels[0,0] = 1
    if n_qubits > 1:
      state_labels[-1] = -1/np.sqrt(2)
    else:
      state_labels[-1,-1] = 1

  # initialize weights
  np.random.seed(100) #to make results reproducible
  weights = np.random.rand(n_qubits,layers,3)

  if plot:
    plt.scatter(x_train[:,0][y_train==0], x_train[:,1][y_train==0], marker='o',
                color='red', label='class 0')
    plt.scatter(x_train[:,0][y_train==1], x_train[:,1][y_train==1], marker='*',
                color='blue', label='class 1')
    plt.legend()
    plt.show()

  return (x_train,y_train),(x_test,y_test),state_labels,weights

```

```python
(x_train,y_train),(x_test,y_test),State_labels,params = cerate_data(n_qubits,6,'moons',900,test_size=0.1)
```

```python
State_labels[0],x_train.shape,params
```

    (array([1.+0.j, 0.+0.j]),
     (810, 3),
     array([[[0.54340494, 0.27836939, 0.42451759],
             [0.84477613, 0.00471886, 0.12156912],
             [0.67074908, 0.82585276, 0.13670659],
             [0.57509333, 0.89132195, 0.20920212],
             [0.18532822, 0.10837689, 0.21969749],
             [0.97862378, 0.81168315, 0.17194101]]]))

state labels are of shape 2 since we have only one qubit state and it's represented by a column vetor of two entries

```python
def circuit(params,X,State_labels=State_labels):

    """

    this function takes 3 inputs:

    params: array like that determines the number of qubits and the number of layers
    X: a single sample that contains the features
    State_labels: are the target states created from create_data()
    it returns the fidelities from the two target states in a list

    """

    n_qubits = len(params)
    layers = len(params[0])
    qc = QuantumCircuit(n_qubits)

    n_features = len(X)

    for layer in range(layers):

        for qubit in range(n_qubits):

            for index in range(0,n_features,3):
              qc.u3(X[index],X[index+1],X[index+2],qubit)

            qc.u3(params[qubit][layer][0],params[qubit][layer][1],params[qubit][layer][2],qubit)

        if n_qubits > 1 and (layer < layers-1):
          qc.cz(0,1)

    backend_sim = Aer.get_backend('statevector_simulator')

    job_sim = execute(qc, backend_sim)

    result_sim = job_sim.result()

    state_vector = result_sim.get_statevector(qc)
    fid_1 = state_fidelity(state_vector,State_labels[0])
    fid_2 = state_fidelity(state_vector,State_labels[1])
    #print(average_data(counts,vals))
    return [fid_1,fid_2]
```

```python
def cost(params,X,y,State_labels):
    """
    params: array like that determines the number of qubits and the number of layers

    X: samples that contain the features

    y: are the labels for that sample

    State_labels: are the target states created from create_data()

    this function returns the binary cross entropy cost from the computed fidelities and corresponding labels
    """
    value = []

    for i in range(len(X)):
        value.append(circuit(params,X[i],State_labels))

    res = log_loss(y,value,labels=y_train)
   # print(res)
    return res

```

### Quantum Gradients

[Quantum gradients](https://arxiv.org/abs/1811.11184) can be computed by the following equation:

\begin{align}
  \frac{df}{d\theta }\,=\,\frac{1}{2}\,\left( f\left( x,\theta +\frac{\pi }{2} \right)-f\left( x,\theta -\frac{\pi }{2} \right) \right)\\ \\
\end{align}

where $x$ is the input data, $\theta$ is the weights and $f$ is quantum unitary gate.

```python
def gradient(params,x,y,State_labels):
    """
    this function computes the gradients for each varaible in the quantum circuit
    it takes:
    x: a single sample
    y: the corresponding label
    State_labels: are the target states created from create_data()
    """
    g = np.zeros(params.shape)

    for qubit in range(len(params)):

        for layer in range(len(params[qubit])):

            for val in range(len(params[qubit][layer])):

                params[qubit][layer][val] = params[qubit][layer][val]+np.pi/2

                F1 = cost(params,x,y,State_labels)

                params[qubit][layer][val] = params[qubit][layer][val]-np.pi/2

                params[qubit][layer][val] = params[qubit][layer][val]-np.pi/2

                F2 = cost(params,x,y,State_labels)

                params[qubit][layer][val] = params[qubit][layer][val]+np.pi/2

                g[qubit][layer][val] = (F1-F2)/2
    return g
```

```python
def optimize(params,X,y,lr,State_labels):
    """

    this fucntion controls SGD by computing the gradients and updating the weights

    """
    g = gradient(params,X,y,State_labels)

    new_weights = params - (lr*g)

    return new_weights
```

```python
def predict(params,X):

    """
    this function takes only the weights "params" and a single sample from the data

    it returns a continous output ranging from 0 to 1. this requires rounding the output

    """

    fid_1,fid_2 = circuit(params,X)

    if fid_1 > fid_2:

      return abs(1-fid_1)

    else:

      return fid_2

```

I have saved you the time and the following are the weights and the testing

```python
best_weights = np.array([[[ 1.13449496,  0.28881491,  1.41365572],
        [ 1.04005053,  1.11646599, -1.71870497],
        [ 0.23246252, -0.40714122,  0.48893466],
        [ 1.21158487,  0.71205549, -0.76677812],
        [ 2.46779094, -0.92909167, -0.34517346],
        [-0.45782809,  0.81168315, -0.0658338 ]]])

pred = []

for j in range(len(x_test)):
    pred.append(predict(best_weights,x_test[j]))
```

```python
print(accuracy_score(np.array(pred).round(),y_test))
print(classification_report(np.array(pred).round(),y_test))
```

    1.0
                  precision    recall  f1-score   support

             0.0       1.00      1.00      1.00        45
             1.0       1.00      1.00      1.00        45

        accuracy                           1.00        90
       macro avg       1.00      1.00      1.00        90
    weighted avg       1.00      1.00      1.00        90

```python
xx, yy = np.meshgrid(np.linspace(-2, 2, 50), np.linspace(-2, 2, 50))
X_grid = [np.array([x, y]) for x, y in zip(xx.flatten(), yy.flatten())]
X_grid = np.hstack((X_grid,np.zeros(len(X_grid)).reshape(-1,1)))
predictions_grid = [predict(best_weights, x) for x in X_grid]

Z = np.reshape(np.array(predictions_grid).round(), xx.shape)
cm = plt.cm.RdBu
cnt = plt.contourf(xx, yy, Z, levels=np.arange(0., 1.1, 0.1), cmap=cm, alpha=.8)

plt.colorbar(cnt, ticks=[0, 0.5, 1])
plt.scatter(x_train[:, 0][y_train==0], x_train[:, 1][y_train==0], c='g', marker='^', edgecolors='k',label='train 0')
plt.scatter(x_train[:, 0][y_train==1], x_train[:, 1][y_train==1], c='b', marker='*', edgecolors='k',label='train 1')
plt.scatter(x_test[:, 0][y_test==0], x_test[:, 1][y_test==0], c='r', marker='+', edgecolors='k',label='test 0')
plt.scatter(x_test[:, 0][y_test==1], x_test[:, 1][y_test==1], c='yellow', marker='x', edgecolors='k',label='test 1')

plt.ylim(-1.07, 1.07)
plt.xlim(-1.07, 1.07)
plt.legend()
plt.show()

```

The following cell of course gives you the pleasure of waiting till tyou train the network

```python
np.random.seed(100) #to be reproducible
#params = np.random.rand(1,6,3)
weights = params
loss = np.inf
lr = 0.22
test_loss = []
epochs = 500
batch_size = 20

for it in range(epochs):

    # create random indices every iteration "stochastic gradient descent way not mini batch because of long time"
    index = np.random.randint(0,len(x_train),batch_size)
    x_batch,y_batch = x_train[index],y_train[index]
    # if you want to decay lr uncomment the following and edit it the way you want
    #if i+1 == 500 or i+1 == 800:
     #   lr = lr - 0.05
    weights = optimize(weights,x_batch,y_batch,lr,State_labels)

    res = cost(weights,x_test,y_test,State_labels)
    test_loss.append(res)

    # save best weights based on testing portion of the data
    if res < loss:
        loss = res
        var = weights
    print(
        "Epoch: {:2d} | testing loss: {:4f}".format(it+1,res))
```

```python
plt.xlabel('no of epochs')
plt.ylabel('test loss')
plt.plot(range(len(test_loss)),test_loss)
plt.show()
```

## Important Resources

1. [qiskit book](https://qiskit.org/textbook/preface.html) which includes aqua the main source of AI algorithms in Qiskit
2. [quantum machine learning book by Prof. Peter Wittek](https://www.researchgate.net/publication/264825604_Quantum_Machine_Learning_What_Quantum_Computing_Means_to_Data_Mining) may god rest his soul
3. [Introduction to quantum machine learning ](https://arxiv.org/pdf/1409.3097.pdf)
4. [Supervised Learning with Quantum Computers](https://www.springer.com/gp/book/9783319964232)
5. [Quantum Computing for computer engineers](https://www.youtube.com/watch?v=F_Riqjdh2oM)
6. [Quantum machine learning libraries](https://pennylane.ai/)

---

## Possible developments for this notebook
1. Instead of measuring the fidelity we can use a real quantum computer and from the counts of of the states we can do a hermitian measurement which is the same as measuring the fidelity. This line is commented in the circuit function. Be careful because you should increase the number of shots while doing so to avoid any vairance in the output

2. For this to happen we will need to convert the label vector into a matrix this can be done using the following code:

```
np.outer(zero,zero)
```
3. instead of indexing the paramters to calculate the gradients in this way we can unpack them and and only use a single for loop which can boost the speed a little bit.
4. You can also try to smooth out the state labels and check what would happen to cost function.


# pyhawkes

## Introduction

Hawkes processes are a mathematical tool extending a system of simple Poisson processes by allowing individual counting processes to be interdependent. These inderdependencies between individual processes manifest through self-excitation and cross-excitation effects of an occurrence of an event on the conditional intensity function.

In mathematical terms, the definition of na D-variate Hawkes process is the following:

Let us have a filtration $\mathbb{F}^{d} = \{\mathscr{F}^{d}_t\}_{t\geq0}$, where $\mathscr{F}^{d}_{t}$ is a $\sigma-$algebra generated by a simple non-explosive counting process $N^{d}(t)$. Then let us define $\mathbb{F}^{'} = \{\mathscr{F}_{t}^{'}\}_{t\geq0}$, where $\mathscr{F}_{t}^{'} = \bigcup_{d=1}^{D} \mathscr{F}^{d}_{t}$.

A mutivariate Hawkes process is a vector $\mathbf{N(t)} = (N^{1}(t),...,N^{D}(t))$ of such counting processes satisfying $\forall d \in \{1,..., D\}$:

1. $N^{d}(0) = 0$

2. Conditional intensity function $\lambda^{d}(t|\mathscr{F}_{t}^{'})$ is an $\mathscr{F}_{t}^{'}$-predictable process of the form:
\begin{equation}
\lambda^{d}(t|\mathscr{F}_{t}^{'}) = \mu_d + \sum_{i = 1}^{D} \int_{0}^{t}g^{(d,i)}(t-u)dN^{i}(u),
\end{equation}
where the integral is a Stieltjes integral and $\forall (c, d) \in B$, where $B = \{ (i,j): i \in \{1,...,D\}$, $j \in \{1,...,D\}\}, g^{(c, d)}(t): \mathbb{R}^{+}_{0} \rightarrow \mathbb{R}^{+}_{0}$.

3. $P(N^{d}(t + h) - N^{d}(t) = 1 |\mathscr{F}_{t}) = \lambda^{d}(t|\mathscr{F}_{t}^{'})h + o(h)$
4. $P(N^{d}(t + h) - N^{d}(t) \geq 2 | \mathscr{F}_{t}) = o(h)$,

where $o(h)$ represents small-o-notation.

The condition of positivity of parameters is usually applied. Also, for an D-variate Hawkes process to be stationary, the matrix $A$ of size $DxD$ with entries $a_{c,d} = \int_{0}^{\infty} g^{c,d}(v) dv$ needs to have a spectral radius strictly less than 1.

## Kernels

Exponential kernel of the form $g(t|\alpha, \beta) = \alpha e^{-\beta t}$ is the most commonly used one, as it allows for $O(n)$ reduction of the usual $O(n^2)$ complexity of the computation of log-likelihood, compensator or simulation of the process through the use of recursive formula. $\alpha$ parameter represents the measure of an instantaneous impact of the event, while $\beta$ parameter represents the speed of decay of the impact.

Another popular kernel is a power-law kernel that takes the form $\frac{\alpha}{(t + \gamma)^{\beta}}$, where $\beta > 1$ . This kernel is however prohibitively slow due to its $O(n^2)$ complexity. Therefore, we use parametrization composed of power-law factors in sum of exponentials as provided by [2]:

$$
g(t|n,\epsilon,\tau_0) = \frac{n}{Z} \Big(\sum_{i = 0}^{M-1}\xi_{i}^{-(1+\epsilon)}e^{-\frac{t}{\xi_i}}\Big),
$$

where $\xi_i = \tau_0 m^i$ for $0 \leq i < M$. Parameter Z is chosen such that $\int_{0}^{\infty}g(t)dt = n$, therefore $Z = \sum_{i=0}^{M-1}(\tau_0 m^i)^{-\epsilon}$. Parameter $m$ controls the precision of the approximation and $M$ specifies the range of it. $1 + \epsilon$ term approximates the power-law decay with an identical exponent.

## Simulation

As a simulation method, we use Ogata's thinning method. The basic idea is that after an event in the multivariate Hawkes process, the process behaves likes an inhomogeneous Poisson process. Therefore, the simulation successively simulates first points for a series of inhomogeneous Poisson processes. For a more detailed overview, see [1]. Functions sim_[type]_hawkes (where type may be exp, power or gen) return simulated series of the D-dimensional Hawkes process.

## Goodness-of-fit tests

The goodness-of-fit is mostly done for Hawkes processes through the application of a random time change theorem [1], [3]. In short, we may transform the original Hawkes process series to a new series that should behave like a sequence of i.i.d. exponential r.v.s with unit rate. The transformation takes the form:

$$
V^{d}_{i} = \int_{T^{d}_{i}}^{T^{d}_{i+1}} \lambda^d(t|\mathscr{F}_{t}^{'})dt,
$$

where $T^{d}_{i + 1}$ and $T^{d}_{i}$ are jump times of the dimension d of the original Hawkes process. Then, $\{V^{d}_{i}\}_{i = 1, ..., N^{d}(T)}$ is a sequence of i.i.d. exponential r.v.s with rate 1. Kolmogorov-Smirnov test and Excess Dispersion test [4] are used to test the exponential property and Ljung-Box or Box-Pierce tests are used to test the independence property. The functions comp_[type]_hawkes (where type may be exp, power or gen) return the sequences $\{V^{d}_{i}\}_{i = 1, ..., N^{d}(T)}$ for all D.

## Log-likelihood

The log-likelihood of an D-variate Hawkes process takes the following form [1]:

$$
log L = \sum_{d=1}^{D} \Big[ \sum_{i:T_{i}^d < T}^{} log(\lambda^{d}(T_{i}^{d}|\mathscr{F}_{T_i}^{'})) - \int_{0}^{T}\lambda^{d}(u|\mathscr{F}_{u}^{'})du) \Big]
$$

The functions lik_[type]_hawkes (where type may be exp, power or gen) return the value of log L.

## Literature

[1] For a more detailed description, please refer to notes prepared by Yuanda Chen, https://www.math.fsu.edu/~ychen/hawkes.html and references provided there.

[2] LALLOUACHE, Mehdi; CHALLET, Damien. The limits of statistical significance of Hawkes processes fitted to financial data. *Quantitative Finance*, 2016, 16.1: 1-11.

[3] BOWSHER, Clive G. Modelling security market events in continuous time: Intensity based, multivariate point process models. *Journal of Econometrics*, 2007, 141.2: 876-912.

[4] ENGLE, Robert F.; RUSSELL, Jeffrey R. Autoregressive conditional duration: a new model for irregularly spaced transaction data. *Econometrica*, 1998, 1127-1162.

```python

```


```python
# Imports
import torch
cuda = torch.cuda.is_available()
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
import sys
sys.path.append("../../semi-supervised")
```

# Variational Autoencoder

The variational autoencoder (VAE) was described in its current form in [Kingma 2013](https://arxiv.org/abs/1312.6114). The model consists of an encoder/inference network $q_{\phi}(z|x)$ and a decoder/generative network $p_{\theta}(x|z)$. The main idea is that it is possible to both reconstruct and generate samples from from some input distribution by learning a variational distribution over the latent variable $z$.

The VAE therefore has a bottleneck structure, where the input $x$ is encoded into a latent variable $z$. New data can then be generated by feeding a latent code into the generator network - $\widehat{x} \sim p_{\theta}(z|x)$. The diagram above shows the generative model (right) and how the latent variable $z$ is inferred from $x$ (left).

Below we will instantiate a new variational autoencoder with this bottleneck structure consisting of a 2-layer encoder network turning an input MNIST image into a latent code: $784 \to 256 \to 128 \to 32$. We also have a decoder that performs the operation in reverse: $32 \to 128 \to 256 \to 784$.

```python
from models import VariationalAutoencoder
from layers import GaussianSample
model = VariationalAutoencoder([784, 32, [256, 128]])
model
```

    VariationalAutoencoder(
      (encoder): Encoder(
        (hidden): ModuleList(
          (0): Linear(in_features=784, out_features=256)
          (1): Linear(in_features=256, out_features=128)
        )
        (sample): GaussianSample(
          (mu): Linear(in_features=128, out_features=32)
          (log_var): Linear(in_features=128, out_features=32)
        )
      )
      (decoder): Decoder(
        (hidden): ModuleList(
          (0): Linear(in_features=32, out_features=128)
          (1): Linear(in_features=128, out_features=256)
        )
        (reconstruction): Linear(in_features=256, out_features=784)
        (output_activation): Sigmoid()
      )
    )

Notice how the middle most layer consists of a `GaussianSample` layer, in which we turn the input digit into the parameters of a Normal distribution with parameters $\mu$ and $\sigma$. This allows us to use the *reparametrization trick* to sample from this distribution to introduce stochasticity into the network.

```python
from torch.autograd import Variable

gaussian = GaussianSample(10, 1)
z, mu, log_var = gaussian(Variable(torch.ones(1, 10)))

print(f"sample {float(z.data):.2f} drawn from N({float(mu.data):.2f}, {float(log_var.exp().data):.2f})")
```

    sample -1.56 drawn from N(-0.38, 1.48)

## Training

How do we go about training a variational autoencoder then? We want to model the data distribution $p(x)$, from here we can introduce a variational distribution $q(z|x)$ by multiplying and dividing by this distribution. Now we employ Jensen's inequality to move the logarithm inside the integral, we can do this because $\log$ is concave and because $q(z|x)$ is a probability distribution. From here on we just rearrange and we see that a lower bound on the marginal probability of the data $p(x)$ is just an expectation over the likelihood of the data minus the KL-divergence between the variational distribution and a prior $p(z)$.

\begin{align}
\log p(x) &= \log \int p(x, z) \ dz = \log \int q(z|x) \frac{p(x, z)}{q(z|x)} \ dz\\
          &\geq \int q(z|x) \log \frac{p(x, z)}{q(z|x)} \ dz = \int q(z|x) \log p(x|z) + \log \frac{p(z)}{q(z|x)} \ dz\\
          &= \int q(z|x) \log p(x|z) \ dz + \int q(z|x) \log \frac{p(z)}{q(z|x)} \ dz\\
          &= \mathbb{E}_{q(z|x)} [\log p(x|z)] - KL(q(z|x)||p(z)) = \mathcal{L}(x)
\end{align}

To make things even more concrete, we show how we can go from this equation to an actual algorithm. Recall that the expectation is just an arithmetic mean, which can be approximated using Monte-Carlo samples. In fact for most applications we can do with just a single sample, even though this provides infinite variance.

$$\mathbb{E}_{q(z|x)} [\log p(x|z)] = \lim_{N \to \infty} \frac{1}{N} \sum_{i=1}^{N} \log p(x_i|z_i) \approx \frac{1}{M} \sum_{i=1}^{M} \log p(x_i|z_i)$$

As you can see the likelihood is just the log probability of the data given the latent variable, but the latent variable is itself derived from the data - we can just use the reconstruction error! In the MNIST case, it is most fitting to use the Bernoulli / binary cross entropy.

Finally, the second term is the Kullback-Leibler divergence. It states that whatever distribution we learn over $q(z|x)$ can never be very far from the prior $p(z)$. This is both good and bad news. Ideally we want a reconstruction that is as good as possible, i.e. only relying on the likelihood, which will only occur if $q(z|x) = p(z)$. This will never happen in practice as the q-distribution will be very complex in order to produce latent variables that result convincing samples. On the plus side, the KL-term acts as a regularizer that pulls the distribution towards the prior, which is the whole reason why we can create samples.

This term can either be computed analytically, or by sampling similarly to the way we did it in the expectation, which you can see in the printed doc string below.

```python
print(model._kld.__doc__)
```

            Computes the KL-divergence of
            some element z.

            KL(q||p) = -∫ q(z) log [ p(z) / q(z) ]
                     = -E[log p(z) - log q(z)]

            :param z: sample from q-distribuion
            :param q_param: (mu, log_var) of the q-distribution
            :param p_param: (mu, log_var) of the p-distribution
            :return: KL(q||p)

```python
from datautils import get_mnist

_, train, validation = get_mnist(location="./", batch_size=64)

# We use this custom BCE function until PyTorch implements reduce=False
def binary_cross_entropy(r, x):
    return -torch.sum(x * torch.log(r + 1e-8) + (1 - x) * torch.log(1 - r + 1e-8), dim=-1)

optimizer = torch.optim.Adam(model.parameters(), lr=3e-4, betas=(0.9, 0.999))
```

```python
for epoch in range(50):
    model.train()
    total_loss = 0
    for (u, _) in train:
        u = Variable(u)

        if cuda: u = u.cuda(device=0)

        reconstruction = model(u)

        likelihood = -binary_cross_entropy(reconstruction, u)
        elbo = likelihood - model.kl_divergence

        L = -torch.mean(elbo)

        L.backward()
        optimizer.step()
        optimizer.zero_grad()

        total_loss += L.data[0]

    m = len(train)

    if epoch % 10 == 0:
        print(f"Epoch: {epoch}\tL: {total_loss/m:.2f}")
```

## Sampling from the generative model

Now that we have trained the network, we can begin to sample from it. We simply give it some random noise distributed according to the prior $p(z) = \mathcal{N}(0, I)$ and send it through the decoder. This process generates a slew of samples that look like they come from the original distribution $p(x)$.

```python
model.eval()
x_mu = model.sample(Variable(torch.randn(16, 32)))
```

```python
f, axarr = plt.subplots(1, 16, figsize=(18, 12))

samples = x_mu.data.view(-1, 28, 28).numpy()

for i, ax in enumerate(axarr.flat):
    ax.imshow(samples[i])
    ax.axis("off")
```

## Side note: Normalizing flows

We can get a more expressive variational approximation by using flows. In the regular variational autoencoder we are limited to a diagonal Gaussian distribution as the output of our encoder, but we can do better. Introducing a full-rank covariance matrix would be an option, but we will quickly see that it becomes computationally infeasible.

What we instead can do is to use change-of-variables to calculate the probability of a random sample under some transformation. In fact, if this transformation is made in a specific way, it is simple to calculate the probability of a sample under a series of $K$ transforms.

$$\ln q_K(z_K) = \ln q_0(z_0) − \sum_{k=1}^{K} \ln \bigg | \det \frac{\partial f_k}{\partial z_{k-1}} \bigg |$$

This is the essence of [Normalizing Flows (Rezende & Mohammed, 2015)](https://arxiv.org/abs/1505.05770). This process increases the expressability of the variational distribution and will likely give a better estimate of the marginal likelihood of the data $p(x)$. However, the parameters of each flow must be learned, meaning that this process will take a long time in order to perform well.

```python
from layers import NormalizingFlows

# Add a series of 16 normalizing flows to the model
# You can do this to *all* of the models in the library.
flow = NormalizingFlows(32, n_flows=16)
model.add_flow(flow)
```

```python
for epoch in range(50):
    model.train()
    total_loss = 0
    for (u, _) in train:
        u = Variable(u)

        if cuda: u = u.cuda(device=0)

        reconstruction = model(u)

        likelihood = -binary_cross_entropy(reconstruction, u)
        elbo = likelihood - model.kl_divergence

        L = -torch.mean(elbo)

        L.backward()
        optimizer.step()
        optimizer.zero_grad()

        total_loss += L.data[0]

    m = len(train)

    if epoch % 10 == 0:
        print(f"Epoch: {epoch}\tL: {total_loss/m:.2f}")
```

## Additional tips

If you want to work with a problem in which the data distribution $x$ is continuous, you can do so by changing the output activation of the decoder and using a different likelihood function, as shown below.

```python
import torch.nn as nn

model.decoder.output_activation = nn.Softmax()
loss = nn.CrossEntropyLoss()
```