```python
import sympy
sympy.init_printing()
%matplotlib inline

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import control

import scipy.optimize
from tbcontrol.responses import fopdt, sopdt
from ipywidgets import interact

def resultplot1(K, tau, theta, y0):
    plt.figure(figsize=(21, 7))
    plt.scatter(ts, ym)
    plt.plot(ts, fopdt(ts, K, tau, theta, y0), color='red')
    plt.grid(True)

def resultplot2(K, tau, zeta, theta, y0):
    plt.figure(figsize=(21, 7))
    plt.scatter(ts, ym)
    plt.plot(ts, sopdt(ts, K, tau, zeta,theta, y0), color='red')
    plt.grid(True)

```

# PID

El controlador PID se compone de:
- Una acción Proporcional para brindar velocidad de respuesta y disminuir el error de estado estable
- Una acción Integral para eliminar el error de estado estable
- Una acción Derivativa para encargarse de errores cuando hay un cambio en el valor absoluto del error teniendo en cuenta que si el error es constante, solo actúan los modos proporcional e integral.

La función de transferencia del PID es:

$$K_C\left ( 1 + \frac{1}{\tau_I s}+ \tau_D s \right )$$

La forma anterior no es realizable debido a la acción derivativa. Por esta razon, es común usar la forma:

$$K_C\left ( 1 + \frac{1}{\tau_I s}+ \frac{\tau_D s}{\alpha \tau_I s+1} \right )$$

donde se escoge $\alpha$ de manera que la parte derivativa sea una "buena" aproximación de la derivada.

## Diseño directo

Esta técnica parte del supuesto de definir el comportamiento deseado para el sistema en lazo cerrado, escribir una función de transferencia de lazo cerrado de una forma apropiada y despejar el compensador

Partiendo del diagrama anterior, se define la relación deseada para $\frac{Y}{Y_{SP}}$ considerando $D=0$. Después se despeja $G_C$.

```python
s, G_C = sympy.symbols('s, G_C')
G_C
```

Se parte de un modelo de primer orden más tiempo muerto como expresión deseada de $\frac{Y}{Y_{SP}}$.

```python
tau_c, phi = sympy.symbols('tau_c, phi', positive=True, nonzero=True)
desired_Y_over_Y_sp = sympy.exp(phi*s)/(tau_c*s + 1)
desired_Y_over_Y_sp
```

Recuerde la forma de la respuesta del sistema ante un escalón.

```python
from ipywidgets import interact
t = sympy.Symbol('t', positive=True)
def plotresponse(theta=(0, 3.), tau_c_in=(1., 5.)):
    desired_response = sympy.inverse_laplace_transform(desired_Y_over_Y_sp.subs({phi: -theta, tau_c: tau_c_in})/s, s, t)
    p = sympy.plot(desired_response, (t, 0, 10), show=False)
    p2 = sympy.plot(1, (t, 0, 10), show=False)
    p.append(p2[0])
    p.show()
interact(plotresponse);
```

    interactive(children=(FloatSlider(value=1.5, description='theta', max=3.0), FloatSlider(value=3.0, description…

A partir de un modelo aproximado para el proceso $\widetilde{G_P}$, puede calcularse la función de transferencia de lazo cerrado.

```python
GPtilde = sympy.Symbol(r'\widetilde{G_P}')
actual_Y_over_Y_sp = GPtilde*G_C/(1 + GPtilde*G_C)
actual_Y_over_Y_sp
```

En este punto, podría encontrarse el compensador al resolver la igualdad con la función de transferencia  deseada para el sistema en lazo cerrado.

```python
G_C_solved, = sympy.solve(desired_Y_over_Y_sp - actual_Y_over_Y_sp, G_C)
G_C_solved
```

Esta función de transferencia tiene $e^{\phi s}$ en el numerador y en el denominador.
- El significado de la exponencial en el numerador es de desplazamiento temporal (tiempo muerto).
- El denominador se aproxima mediante una expansión de serie de Taylor de primer orden.

```python
denom = sympy.denom(G_C_solved)
G_C_rational = G_C_solved*denom/denom.subs(sympy.exp(phi*s), 1 + phi*s)
G_C_rational
```

Se define la forma para el compensador PID

$$K_C\left ( 1 + \frac{1}{\tau_I s}  + \tau_D s      \right )$$

```python
K_C, tau_I, tau_D = sympy.symbols('K_C, tau_I, tau_D', positive=True, nonzero=True)
PID = K_C*(1 + 1/(tau_I*s) + tau_D*s)
PID.expand().together()
```

Aunque la forma anterior no es realizable debido a la acción derivativa. Por esta razon, es común usar la forma:

$$K_C\left ( 1 + \frac{1}{\tau_I s}+ \frac{\tau_D s}{\alpha \tau_I s+1} \right )$$

```python
alpha = sympy.symbols('alpha')
ISA = K_C*(1 + 1/(tau_I*s) + tau_D*s/(alpha*tau_D*s + 1))
ISA.expand().together().simplify()
```

El objetivo es hallar los parámetros del compensador para satisfacer la expresión racional $G_C$

```python
G_C_rational
```

Se requiere un modelo de proceso $G$. En este ejemplo se usará un modelo de segundo orden con tiempo muerto.

```python
K, tau, zeta, phi,= sympy.symbols('K, tau, zeta, phi', positive=True)
G2 = K*sympy.exp(phi*s)/((tau*s)**2 + 2*tau*zeta*s + 1)
G2
```

Se incorpora el modelo del sistema a la expresión del compensador.

```python
target_G_C = G_C_rational.subs(GPtilde, G2).expand().together()
target_G_C
```

Se buscan los parámetros del PID que conformen el compensador despejado.

```python
# diferencia entre compensadores determinados
zeroeq = (target_G_C - PID).simplify()
zeroeq
```

```python
numer, denom = zeroeq.as_numer_denom()
eq = sympy.poly(numer, s)
eqs = eq.coeffs()
sympy.solve(eqs, [K_C, tau_D, tau_I])
```

La solución obtenida muestra:

$$K_C = -\frac{2\tau \zeta}{K(\phi-\tau_C)}$$
$$\tau_D = \frac{\tau}{2\zeta}$$
$$\tau_I = 2\tau\zeta$$

Los parámetros del compensador quedan completamente definidos a partir de un modelo aproximado del proceso

$$\frac{K}{(\tau s)^2 + 2\tau \zeta s + 1} \cdot e^{s\phi}$$

Y una respuesta deseada

$$\frac{e^{\phi s}}{\tau_c s + 1}$$

**Ejemplo**

Ajuste un controlador PID para un sistema que fue excitado con un escalón unitario y su respuesta fue registrada.

```python
GP = control.tf([1, 2], [2, 3, 4, 1])
GP
df = pd.read_csv('Datos\DatoEscalon.csv')
df.head()
```

      Unnamed: 0
      tiempo
      U
      Y

      0
      0
      0.000000
      1.0
      5.011953

      1
      1
      0.196392
      1.0
      5.050839

      2
      2
      0.392785
      1.0
      5.101922

      3
      3
      0.589177
      1.0
      4.996570

      4
      4
      0.785569
      1.0
      5.060810

```python
ym = df.Y
ts = df.tiempo
interact(resultplot2, K=(0.0, 10.0), tau=(0., 10.),
         theta=(0., 10.), zeta = (0, 10.0), y0=(0., 10.));
```

    interactive(children=(FloatSlider(value=5.0, description='K', max=10.0), FloatSlider(value=5.0, description='t…

```python
p_inicial = [2, 1, 1.1, 0.5, 4.9]
[K_1, tau_1, zeta_1, theta_1, y0_1], pcov = scipy.optimize.curve_fit(sopdt, ts, ym, p_inicial)
[K_1, tau_1, zeta_1, theta_1, y0_1]
```

Se define el comportamiento deseado.

```python
tauC = 1.2
```

Se calculan los parámetros del PID

```python
sympy.solve(eqs, [K_C, tau_D, tau_I])
```

```python
KC1 = -2*tau_1*zeta_1/(K_1*(-theta_1 - tauC))
TD1 = tau_1/(2*zeta_1)
TI1 = 2*tau_1*zeta_1
[KC1,TI1,TD1]
```

```python
PID1 = KC1*(1 + 1/(TI1*s) + TD1*s)
PID1.expand().together()
```

```python
GC_PID1 = KC1*(1 + control.tf(1,[TI1,0]) + control.tf([TD1,0],1))
GC_PID1
```

$$\frac{0.02943 s^2 + 1.931 s + 0.7193}{2.685 s}$$

```python
_, yLA = control.step_response(GP,ts)
_, yLC1 = control.step_response(control.feedback(GP,1),ts)
_, yLCPID1 = control.step_response(control.feedback(GC_PID1*GP,1),ts)

plt.plot(ts,yLA,
         ts,yLC1,
         ts,yLCPID1)
plt.legend(('Lazo abierto','Realimentado','con PID directo'))
plt.grid()

```

Observe que se presentan oscilaciones aunque la respuesta deseada se modeló como un sistema de primer orden con tiempo muerto.

- ¿A qué se debe esta discrepancia?
- ¿Qué puntos débiles puede tener esta metodología?

------

## Reglas de sintonía

[Ziegler y Nichols](https://controlautomaticoeducacion.com/control-realimentado/ziegler-nichols-sintonia-de-control-pid/) propusieron métodos para sintonizar un controlador PID.

### En lazo abierto
El método está desarrollado para sistemas cuya respuesta ante un escalón unitario tienen forma de $S$., es decir, sin sobreimpulsos ni dinámicas inestables. En este caso, el sistema puede aproximarse mediante un modelo de primer orden y tiempo muerto:

$$\frac{K}{\tau s + 1}e^{-Ls}$$

como se muestra en la figura.

A partir de este modelo, se sugieren los siguientes valores para los compensadores:

| Controlador | $K_C$               | $\tau_I$       | $\tau_D$ |
|-------------|---------------------|----------------|----------|
| P           |$\frac{\tau}{KL}$    | $\infty$       | $0$      |
| PI          |$0.9\frac{\tau}{KL}$ |$\frac{L}{0.3}$ | $0$      |
| PID         |$1.2\frac{\tau}{KL}$ |$2L$            | $0.5L$   |

```python
ym = df.Y
ts = df.tiempo
interact(resultplot1, K=(0.0, 10.0), tau=(0., 10.),
         theta=(0., 10.), y0=(0., 10.));
```

    interactive(children=(FloatSlider(value=5.0, description='K', max=10.0), FloatSlider(value=5.0, description='t…

```python
p_inicial = [1.8, 2.0, 1.0, 5.0]
[K_2, tau_2, theta_2, y0_2], pcov = scipy.optimize.curve_fit(fopdt, ts, ym, p_inicial)
[K_2, tau_2, theta_2, y0_2]
```

A partir del modelo, se calculan los parámetros del PID.

$$K_C = 1.2 \frac{\tau}{KL}$$
$$\tau_I = 2 L$$
$$\tau_D = 0.5 L$$

```python
KC2 = 1.2*tau_2/(K_2*theta_2)
TI2 = 2*theta_2
TD2 = 0.5*theta_2
[KC2,TI2,TD2]
```

```python
GC_PID2 = KC2*(1 + control.tf(1,[TI2,0]) + control.tf([TD2,0],[0.001*TD2,1]))
GC_PID2
```

$$\frac{1.243 s^2 + 3.319 s + 2.217}{0.0005604 s^2 + 1.497 s}$$

```python

_, yLCPID2 = control.step_response(control.feedback(GC_PID2*GP,1),ts)

plt.plot(ts,yLA,
         ts,yLC1,
         ts,yLCPID1,
         ts,yLCPID2)
plt.legend(('Lazo abierto','Realimentado','PID directo','ZyN abierto'))
plt.grid()

```

### En lazo cerrado
El método está desarrollado para sistemas que lleguen a oscilación con al ser realimentados.

Se cierra el lazo eliminando las acciones derivativas e integradoras, es decir, se configura

$$\tau_I = \infty$$
$$\tau_D = 0$$

Se varía la ganancia proporcional hasta obtener una respuesta en lazo cerrado que sea una oscilación de amplitud constante.

- La ganancia del compensador que logra esta respuesta se conoce como ganancia crítica $K_u$.
- El periodo de dicha oscilación es el periodo crítico $P_u$.

Con estos valores se encuentran los parámetros del PID

| Controlador   |	$K_p$   |	 $\tau_I$       |  $\tau_d$     |
|---------------|-----------|-------------------|---------------|
| P	            | $0.5K_u$  | 	$\infty$        |	$0$         |
| PI	        | $0.45K_u$ |$\frac{1}{1.2}P_u$ | 	$0 $        |
| PID	        | $0.6K_u$  | $0.5P_u$          |  $0.125P_u$     |

```python
GP
```

$$\frac{s + 2}{2 s^3 + 3 s^2 + 4 s + 1}$$

```python
def resultplotKfeed(K):
    plt.figure(figsize=(21, 7))
    ts = np.linspace(0,5,1000)
    _, yLCPIDk = control.step_response(control.feedback(K*GP,1),ts)
    plt.plot(ts,yLCPIDk)
    plt.grid(True)

interact(resultplotKfeed, K=(0.0, 20.0))
```

    interactive(children=(FloatSlider(value=10.0, description='K', max=20.0), Output()), _dom_classes=('widget-int…

    <function __main__.resultplotKfeed(K)>

```python
Ku = 9.4
Pu = 2.4
KC3 = 0.6*Ku
TI3 = 0.5*Pu
TD3 = 0.125*Pu
GC_PID3 = KC3*(1 + control.tf(1,[TI3,0]) + control.tf([TD3,0],1))
GC_PID3
```

$$\frac{2.03 s^2 + 6.768 s + 5.64}{1.2 s}$$

```python
_, yLCPID3 = control.step_response(control.feedback(GC_PID3*GP,1),ts)
plt.figure(figsize=(14, 7))

plt.plot(ts,yLA,
         ts,yLC1,
         ts,yLCPID1,
         ts,yLCPID2,
         ts,yLCPID3)
plt.legend(('Lazo abierto','Realimentado','PID directo','ZyN abierto','ZyN cerrado'))
plt.grid()
```

- ¿Cuál es la función de ransferencia en lazo cerrado desde la Referencia hacia le señal de control?

$$\frac{Y}{Y_{sp}} = \frac{G_C G_P}{1+G_C G_P}$$

$$E = Y_{sp}-Y$$
$$E + Y = Y_{sp}$$
$$Y = Y_{sp}- E$$
$$Y = Y_{sp}- \frac{U}{G_C}$$

$$U = E G_C$$
$$Y = U G_P$$
$$Y_{sp}- \frac{U}{G_C} = U G_P$$
$$Y_{sp} = U G_P +  \frac{U}{G_C}$$
$$Y_{sp} = U (G_P +  \frac{1}{G_C})$$
$$Y_{sp} = U (\frac{G_C G_P}{G_C} +  \frac{1}{G_C})$$
$$Y_{sp} = U (\frac{G_C G_P + 1}{G_C})$$
$$\frac{Y_{sp}}{U} = (\frac{G_C G_P + 1}{G_C})$$

```python
GC_PID2/(1+GP*GC_PID2)
```

$$\frac{0.001394 s^7 + 3.729 s^6 + 15.54 s^5 + 29.01 s^4 + 31.7 s^3 + 18.25 s^2 + 3.319 s}{6.281 \times 10^{-7} s^7 + 0.003357 s^6 + 4.489 s^5 + 8.596 s^4 + 17.67 s^3 + 15.5 s^2 + 6.637 s}$$

- ¿Cómo se comportan los distintos controladores ante una señal escalón?

```python
# código necesario

_, uLCPID1 = control.step_response(control.feedback(sys1=GC_PID2,
                                                    sys2=GP),ts)
plt.figure(figsize=(14, 7))

plt.plot(ts,uLCPID1)
plt.legend(('PID directo'))
plt.grid()
```

## Sintonización basada en criterios integrales

El comportamiento de los lazos de control puede evaluarse mediante la señal de error $e(t) = ref(t) - Y(t)$. A continuación algunos índices basados en el error.

| Índice                                               |        Definición          |
|------------------------------------------------------|------------------------------|
| IAE:Integral del valor absoluto del error            |   $$\int_0^\infty |e(t)|dt$$ |
| ISE:Integral del cuadrado del error                  |   $$\int_0^\infty e(t)^2dt$$ |
| ITAE:Integral de tiempo por valor absoluto del error |   $$\int_0^\infty t|e(t)|dt$$|

Una técnica de diseño consiste en encontrar los parámetros de un compensador PID que minimicen alguno de los índices mencionados con base en un modelo para el sistema. Puede explorar más [aquí](https://www.researchgate.net/publication/260058488_METODOS_DE_SINTONIZACION_DE_CONTROLADORES_PID_QUE_OPERAN_COMO_SERVOMECANISMOS), [aquí](https://www.researchgate.net/publication/260058590_Optimizacion_del_desempeno_de_los_reguladores_y_servomecanismos_PID) y [aquí](https://www.researchgate.net/publication/255641510_SINTONIZACION_DE_CONTROLADORES_PI_Y_PID_UTILIZANDO_LOS_CRITERIOS_INTEGRALES_IAE_E_ITAE)

- Encuentre y simule un controlador PID que minimice el índice ITAE para el sistema

```python
GP
```

$$\frac{s + 2}{2 s^3 + 3 s^2 + 4 s + 1}$$

```python
# código necesario
```

- ¿Cómo se comportan los distintos controladores ante una señal escalón?

```python
# código necesario
```


```python
import matplotlib.pyplot as plt
import numpy as np
import os
from scipy.optimize import curve_fit
import math

%matplotlib inline
```

```python
#os.listdir()

```

```python
filename = "08_01_19-2.scan"
def func(x, a, b, c):
  #return a * np.exp(-b * x) + c
  return a * np.log(b * x) + c

f = open(filename)
no_of_mass = 0
start = False
raw_datas = []
iterations = []
time = []

for line in f:

    if line.find("#mass")>=0:
        no_of_mass += 1
        tmp = line.split(":")[-1].strip()
        iterations.append(int(tmp))

    if not line[0] == "#" and not line == "\n":

        if line.strip() == "ALL:":
            start = True

        if not start:
            tmp = line.split()
            time.append(tmp)

        if start and not line.strip()=="ALL:":
            raw_datas.append(line)

f.close()

time = [float(i) for i in list(zip(*(time)))[0]]

all_datas = [[float(j) for j in i.split()] for i in raw_datas]
data_run_cycle = int(len(all_datas)/sum(iterations))

datas = [[[] for j in range(iterations[i])] for i in range(no_of_mass)]

k = 0
for i in range(no_of_mass):
    for j in range(iterations[i]):
        d = data_run_cycle
        datas[i][j] = all_datas[k:d+k]
        k += d

mass_values = [i[0][0][0] for i in datas]
data_sets = [list(zip(*datas[i])) for i in range(no_of_mass)]

mass_mean_adder, mass_mean, m = [], [], []
varience, varience_collector1, varience_collector2 = [], [], []

for i in range(no_of_mass):
    for k in range(data_run_cycle):
        for j in range(iterations[i]):

            #mass counts
            m_tmp = data_sets[i][k][j][2]
            mass_mean_adder.append(m_tmp)

            #varience
            varience_adder = mass_mean_adder

        m.append(sum(mass_mean_adder)/len(mass_mean_adder))
        mass_mean_adder = []

        #varience
        for v in range(iterations[i]):

            if iterations[i]>1:

                v_tmp = (varience_adder[v]-m[k])**2
                varience_collector1.append(v_tmp)

        if iterations[i]>1:
            varience_collector2.append(sum(varience_collector1)/(len(varience_collector1)-1))
        else: varience_collector2.append(0)

        varience_collector1 = []

    #mass counts
    mass_mean.append(m)
    m = []

    #varience
    varience.append(varience_collector2)
    varience_collector2 = []

#standard deviation
standard_deviation = [[math.sqrt(i) for i in j] for j in varience]

#standard error
standard_error = [[i/math.sqrt(k) for i in j] for j, k in zip(standard_deviation, iterations)]

if len(time)<len(mass_mean[0]):
    for i in range(no_of_mass):
        del mass_mean[i][-1]
        del standard_error[i][-1]

plt.figure(figsize=(10,5), dpi = 100)
deg = 3
for i in range(no_of_mass):
    if iterations[i]>1:
        x, y, err = time, mass_mean[i], standard_error[i]
        lg = "{} : {}".format(mass_values[i], iterations[i])

        plt.errorbar(x, y, yerr = err, fmt = ".", label = lg )

        # polyfit

        z = np.polyfit(x, y, deg)
        p = np.poly1d(z)
        y_fit = [p(i) for i in x]

        plt.plot(x, y_fit, "k-")

        '''# log fit
        popt, pcov = curve_fit(func, x, y)

        #brutal force to avoid errors
        x = np.array(x, dtype=float) #transform your data in a numpy array of floats
        y = np.array(y, dtype=float) #so the curve_fit can work

        yy = func(x, *popt)

        plt.plot(x, yy, 'r-', label = "log fit")'''

'''greek_letterz=[chr(code) for code in range(945,970)]
xs = greek_letterz[10]
plt.title("f(x) = {}".format(p),fontsize=10)
'''
plt.grid(True)
plt.xlabel("Time (ms)")
plt.ylabel("Ion Counts")
plt.legend()
plt.title(filename + ":  Polyfit of Order: %i"%deg)
plt.savefig(filename+".png")
#plt.tight_layout()
plt.show()
plt.close()
```

```python
from scipy.optimize import curve_fit
from sympy import symbols, latex
import numpy as np

x, y, error = time, mass_mean[1], standard_error[1]

def func(x, a, b, c):
  #return a * np.exp(-b * x) + c
  return a * np.log(b * x) + c

popt, pcov = curve_fit(func, x, y)
xs = symbols('\lambda')
tex = latex(func(xs,*popt)).replace('$', '')
plt.title(r'$f(\lambda)= %s$' %(tex),fontsize=16)

#brutal force to avoid errors
x = np.array(x, dtype=float) #transform your data in a numpy array of floats
y = np.array(y, dtype=float) #so the curve_fit can work

yy = func(x, *popt)

plt.figure()

plt.errorbar(x,y, yerr = error, fmt = ".", label = "Orginal")
plt.plot(x, yy, 'r-', label="Fitted Curve")

plt.legend()
plt.show()
```

```python
xs = symbols("\lambda")
print(xs)
```

    \lambda

```python
import sym
sym.Symbols("\lambda")
```

```python
greek_letterz=[chr(code) for code in range(945,970)]
print(greek_letterz)
x = greek_letterz[10]
x
```

    ['α', 'β', 'γ', 'δ', 'ε', 'ζ', 'η', 'θ', 'ι', 'κ', 'λ', 'μ', 'ν', 'ξ', 'ο', 'π', 'ρ', 'ς', 'σ', 'τ', 'υ', 'φ', 'χ', 'ψ', 'ω']

    'λ'

```python
from sympy.utilities.lambdify import implemented_function
#from sympy import sqrt, sin, Matrix
#from sympy import Function
#from sympy.abc import w, x, y, z
x = ['1', '2']
#f = lambdify(x, float(x))
x = np.array(x, dtype = float)
```

```python

```

```python

```

```python

```


```python
import sympy as sym
from sympy.plotting import plot
from sympy import simplify

x = sym.Symbol('x')

gx = sym.sin(sym.pi*x)**2
fx = (gx-1)**3
# plot(fx)
# simplify(fx)
simplify(fx)
plot(fx)
```

포기


# Bayesian Gaussian Mixture

```{note}
In BayesianGaussianMixture, cluster parameters are treated as random variables instead of fixed numbers.

Latent variables $\mathbf{z}$ now includes both the cluster parameters and the cluster assignments.
```

## Examples

```python
"""
manual dataset
"""
import numpy as np
import matplotlib.pyplot as plt

X = np.vstack((np.random.multivariate_normal([0, 0], [[2, 0], [0, 1]], 30),
               np.random.multivariate_normal([4, 5], [[1, 0], [0, 1]], 30)))
np.random.shuffle(X)

plt.scatter(X[:, 0], X[:, 1])
plt.show()
```

```python
from sklearn.mixture import BayesianGaussianMixture

# set n_components large enough
# set weight_concentration_prior small because we have a prior belief there are few clusters
bgm = BayesianGaussianMixture(n_components=10, n_init=10, weight_concentration_prior=0.01)
bgm.fit(X)

# give 0 weights to unnessary clusters
np.round(bgm.weights_, 2)
```

    array([0.51, 0.49, 0.  , 0.  , 0.  , 0.  , 0.  , 0.  , 0.  , 0.  ])

```python
X_predict = np.array([[2, 2], [4, 3]])
bgm.predict(X_predict)
```

    array([0, 1])

## Variational Inference

Bayes' theorem:

$$p(\mathbf{z}|\mathbf{X}) = \mbox{posterior} = \frac{\mbox{likelihood}\times\mbox{prior}}{\mbox{evidence}} = \frac{p(\mathbf{X}|\mathbf{z})p(\mathbf{z})}{p(\mathbf{X})}$$

Unfortunately, the evidence $p(\mathbf{X})$ is often intractable:

$$p(\mathbf{X}) = \int{p(\mathbf{X}|\mathbf{z})p(\mathbf{z})}d{\mathbf{z}}$$

variational inference solves this intractability by picking a family of distribution $q(\mathbf{z};\boldsymbol{\lambda})$.

Then optimize $\boldsymbol{\lambda}$ to make $q(\mathbf{z})$ a good approximation of $p(\mathbf{z}|\mathbf{X})$.

This is achieved by finding the value of $\boldsymbol{\lambda}$ that minimizes the KL divergence from $q(\mathbf{z})$ to
$p(\mathbf{z}|\mathbf{X})$, noted $D_{KL}(q‖p)$.

$$
\begin{equation}
\begin{split}
D_{KL}(q‖p) =& \mathbb{E}_{q}\left[\log\frac{q(\mathbf{z})}{p(\mathbf{z}|\mathbf{X})}\right]\\
=& \mathbb{E}_{q}\left[\log{q(\mathbf{z})} - \log{p(\mathbf{z}, \mathbf{X})} + \log{p(\mathbf{X})}\right]\\
=& \log{p(\mathbf{X})} - \mbox{ELBO}(\mathbf{X};q)
\end{split}
\end{equation}
$$

Minimizing the KL divergence just requires maximizing the ELBO.

In practice, there are different techniques to maximize the ELBO.

E.g *mean field variational inference* and *black box stochastic variational inference(BBSVI)*.


# Simulating Gate Noise
$$
\newcommand{ket}[1]{\left|{#1}\right\rangle}
\newcommand{bra}[1]{\left\langle {#1}\right|}
\newcommand{tr}{\mathrm{Tr}}
$$

## Pure states vs. mixed states

Errors in quantum computing can introduce classical uncertainty in what the underlying state is.
When this happens we sometimes need to consider not only wavefunctions but also probabilistic sums of
wavefunctions when we are uncertain as to which one we have. For example, if we think that an X gate
was accidentally applied to a qubit with a 50-50 chance then we would say that there is a 50% chance
we have the $\ket{0}$ state and a 50% chance that we have a $\ket{1}$ state.
This is called an "impure" or
"mixed"state in that it isn't just a wavefunction (which is pure) but instead a distribution over
wavefunctions. We describe this with something called a density matrix, which is generally an
operator. Pure states have very simple density matrices that we can write as an outer product of a
ket vector $\ket{\psi}$ with its own bra version $\bra{\psi}=\ket{\psi}^\dagger$.
For a pure state the density matrix is simply

$$
   \rho_\psi = \ket{\psi}\bra{\psi}.
$$
The expectation value of an operator for a mixed state is given by

$$
   \langle X \rangle_\rho = \tr{X \rho}
$$
where $\tr{A}$ is the trace of an operator, which is the sum of its diagonal elements
which is independent of choice of basis.
Pure state density matrices satisfy

$$
   \rho \text{ is pure } \Leftrightarrow \rho^2 = \rho
$$
which you can easily verify for $\rho_\psi$ assuming that the state is normalized.
If we want to describe a situation with classical uncertainty between states $\rho_1$ and
$\rho_2$, then we can take their weighted sum
$$
   \rho = p \rho_1 + (1-p) \rho_2
$$
where $p\in [0,1]$ gives the classical probability that the state is $\rho_1$.

Note that classical uncertainty in the wavefunction is markedly different from superpositions.
We can represent superpositions using wavefunctions, but use density matrices to describe
distributions over wavefunctions. You can read more about density matrices [here](https://en.wikipedia.org/wiki/Density_matrix).

# Quantum gate errors
## What are they?
For a quantum gate given by its unitary operator $U$, a "quantum gate error" describes the scenario in which the actually induces transformation deviates from $\ket{\psi} \mapsto U\ket{\psi}$.
There are two basic types of quantum gate errors:

1. **coherent errors** are those that preserve the purity of the input state, i.e., instead of the above mapping we carry out a perturbed, but unitary operation $\ket{\psi} \mapsto \tilde{U}\ket{\psi}$, where $\tilde{U} \neq U$.
2. **incoherent errors** are those that do not preserve the purity of the input state,
    in this case we must actually represent the evolution in terms of density matrices.
    The state $\rho := \ket{\psi}\bra{\psi}$ is then mapped as
    $$
    \rho \mapsto \sum_{j=1}^n K_j\rho K_j^\dagger,
    $$
    where the operators $\{K_1, K_2, \dots, K_m\}$ are called Kraus operators and must obey
    $\sum_{j=1}^m K_j^\dagger K_j = I$ to conserve the trace of $\rho$.
    Maps expressed in the above form are called Kraus maps. It can be shown that every physical map on a finite
    dimensional quantum system can be represented as a Kraus map, though this representation is not generally unique.
    [You can find more information about quantum operations here](https://en.wikipedia.org/wiki/Quantum_operation#Kraus_operators)

In a way, coherent errors are *in principle* amendable by more precisely calibrated control. Incoherent errors are more tricky.

## Why do incoherent errors happen?
When a quantum system (e.g., the qubits on a quantum processor) is not perfectly isolated from its environment it generally co-evolves with the degrees of freedom it couples to. The implication is that while the total time evolution of system and environment can be assumed to be unitary, restriction to the system state generally is not.

**Let's throw some math at this for clarity:**
Let our total Hilbert space be given by the tensor product of system and environment Hilbert spaces:
$\mathcal{H} = \mathcal{H}_S \otimes \mathcal{H}_E$.
Our system "not being perfectly isolated" must be translated to the statement that the global Hamiltonian contains a contribution that couples the system and environment:
$$
H = H_S \otimes I + I \otimes H_E + V
$$
where $V$ non-trivally acts on both the system and the environment.
Consequently, even if we started in an initial state that factorized over system and environment $\ket{\psi}_{S,0}\otimes \ket{\psi}_{E,0}$
if everything evolves by the Schrödinger equation
$$
\ket{\psi_t} = e^{-i \frac{Ht}{\hbar}} \left(\ket{\psi}_{S,0}\otimes \ket{\psi}_{E,0}\right)
$$
the final state will generally not admit such a factorization.

## A toy model
**In this (somewhat technical) section we show how environment interaction can corrupt an identity gate and derive its Kraus map.**
For simplicity, let us assume that we are in a reference frame in which both the system and environment Hamiltonian's vanish $H_S = 0, H_E = 0$ and where the cross-coupling is small even when multiplied by the duration of the time evolution $\|\frac{tV}{\hbar}\|^2 \sim \epsilon \ll 1$ (any operator norm $\|\cdot\|$ will do here).
Let us further assume that $V = \sqrt{\epsilon} V_S \otimes V_E$ (the more general case is given by a sum of such terms) and that
the initial environment state satisfies $\bra{\psi}_{E,0} V_E\ket{\psi}_{E,0} = 0$. This turns out to be a very reasonable assumption in practice but a more thorough discussion exceeds our scope.

Then the joint system + environment state $\rho = \rho_{S,0} \otimes \rho_{E,0}$ (now written as a density matrix) evolves as
$$
\rho \mapsto \rho' :=  e^{-i \frac{Vt}{\hbar}} \rho e^{+i \frac{Vt}{\hbar}}
$$
Using the Baker-Campbell-Hausdorff theorem we can expand this to second order in $\epsilon$
$$
\rho' = \rho - \frac{it}{\hbar} [V, \rho] - \frac{t^2}{2\hbar^2} [V, [V, \rho]] + O(\epsilon^{3/2})
$$
We can insert the initially factorizable state $\rho = \rho_{S,0} \otimes \rho_{E,0}$ and trace over the environmental degrees of freedom to obtain
\begin{align}
\rho_S' := \tr_E \rho' & = \rho_{S,0}  \underbrace{\tr \rho_{E,0}}_{1} - \frac{i\sqrt{\epsilon} t}{\hbar} \underbrace{\left[ V_S \rho_{S,0} \underbrace{\tr V_E\rho_{E,0}}_{\bra{\psi}_{E,0} V_E\ket{\psi}_{E,0} = 0} - \rho_{S,0}V_S  \underbrace{\tr \rho_{E,0}V_E}_{\bra{\psi}_{E,0} V_E\ket{\psi}_{E,0} = 0} \right]}_0 \\
& \qquad - \frac{\epsilon t^2}{2\hbar^2} \left[ V_S^2\rho_{S,0}\tr V_E^2 \rho_{E,0} + \rho_{S,0} V_S^2 \tr \rho_{E,0}V_E^2 - 2 V_S\rho_{S,0}V_S\tr V_E \rho_{E,0}V_E\right] \\
& = \rho_{S,0} - \frac{\gamma}{2} \left[ V_S^2\rho_{S,0} + \rho_{S,0} V_S^2  - 2 V_S\rho_{S,0}V_S\right]
\end{align}
where the coefficient in front of the second part is by our initial assumption very small $\gamma := \frac{\epsilon t^2}{2\hbar^2}\tr V_E^2 \rho_{E,0} \ll 1$.
This evolution happens to be approximately equal to a Kraus map with operators $K_1 := I - \frac{\gamma}{2} V_S^2, K_2:= \sqrt{\gamma} V_S$:
\begin{align}
\rho_S \to \rho_S' &= K_1\rho K_1^\dagger + K_2\rho K_2^\dagger
 = \rho - \frac{\gamma}{2}\left[ V_S^2 \rho + \rho V_S^2\right] + \gamma V_S\rho_S V_S + O(\gamma^2)
\end{align}
This agrees to $O(\epsilon^{3/2})$ with the result of our derivation above. This type of derivation can be extended to many other cases with little complication and a very similar argument is used to derive the [Lindblad master equation](https://en.wikipedia.org/wiki/Lindblad_equation).

# Support for noisy gates on the Rigetti QVM

As of today, users of our Forest API can annotate their QUIL programs by certain pragma statements that inform the QVM that a particular gate on specific target qubits should be replaced by an imperfect realization given by a Kraus map.

## But the QVM propagates *pure states*: How does it simulate noisy gates?
It does so by yielding the correct outcomes **in the average over many executions of the QUIL program**:
When the noisy version of a gate should be applied the QVM makes a random choice which Kraus operator is applied to the current state with a probability that ensures that the average over many executions is equivalent to the Kraus map.
In particular, a particular Kraus operator $K_j$ is applied to $\ket{\psi}_S$
$$
\ket{\psi'}_S = \frac{1}{\sqrt{p_j}} K_j \ket{\psi}_S
$$
with probability $p_j:= \bra{\psi}_S K_j^\dagger K_j \ket{\psi}_S$.
In the average over many execution $N \gg 1$ we therefore find that
\begin{align}
\overline{\rho_S'} & = \frac{1}{N} \sum_{n=1}^N \ket{\psi'_n}_S\bra{\psi'_n}_S \\
& = \frac{1}{N} \sum_{n=1}^N p_{j_n}^{-1}K_{j_n}\ket{\psi'}_S \bra{\psi'}_SK_{j_n}^\dagger
\end{align}
where $j_n$ is the chosen Kraus operator label in the $n$-th trial.
This is clearly a Kraus map itself! And we can group identical terms and rewrite it as
\begin{align}
\overline{\rho_S'} & =
  \sum_{\ell=1}^n \frac{N_\ell}{N}  p_{\ell}^{-1}K_{\ell}\ket{\psi'}_S \bra{\psi'}_SK_{\ell}^\dagger
\end{align}
where $N_{\ell}$ is the number of times that Kraus operator label $\ell$ was selected.
For large enough $N$ we know that $N_{\ell} \approx N p_\ell$ and therefore
\begin{align}
\overline{\rho_S'} \approx \sum_{\ell=1}^n K_{\ell}\ket{\psi'}_S \bra{\psi'}_SK_{\ell}^\dagger
\end{align}
which proves our claim.
**The consequence is that noisy gate simulations must generally be repeated many times to obtain representative results**.

## How do I get started?

1. Come up with a good model for your noise. We will provide some examples below and may add more such
    examples to our public repositories over time. Alternatively, you can characterize the gate under
    consideration using [Quantum Process Tomography](https://arxiv.org/abs/1202.5344) or
    [Gate Set Tomography](http://www.pygsti.info/) and use the resulting process matrices to obtain a
    very accurate noise model for a particular QPU.
2. Define your Kraus operators as a list of numpy arrays `kraus_ops = [K1, K2, ..., Km]`.
3. For your QUIL program `p`, call:
    ```
    p.define_noisy_gate("MY_NOISY_GATE", [q1, q2], kraus_ops)
    ```
    where you should replace `MY_NOISY_GATE` with the gate of interest and `q1, q2` the indices of the qubits.

**Scroll down for some examples!**

```python
import numpy as np
import matplotlib.pyplot as plt
from scipy.stats import binom
import matplotlib.colors as colors
%matplotlib inline
```

```python
from pyquil import Program, get_qc
from pyquil.gates import CZ, H, I, X, MEASURE
from scipy.linalg import expm
```

```python
# We could ask for "2q-noisy-qvm" but we will be specifying
# our noise model as PRAGMAs on the Program itself.
qc = get_qc('2q-qvm')
```

# Example 1: Amplitude damping

Amplitude damping channels are imperfect identity maps with Kraus operators
$$
K_1 = \begin{pmatrix}
1 & 0 \\
0 & \sqrt{1-p}
\end{pmatrix} \\
K_2 = \begin{pmatrix}
0 & \sqrt{p} \\
0 & 0
\end{pmatrix}
$$
where $p$ is the probability that a qubit in the $\ket{1}$ state decays to the $\ket{0}$ state.

```python
def damping_channel(damp_prob=.1):
    """
    Generate the Kraus operators corresponding to an amplitude damping
    noise channel.

    :params float damp_prob: The one-step damping probability.
    :return: A list [k1, k2] of the Kraus operators that parametrize the map.
    :rtype: list
    """
    damping_op = np.sqrt(damp_prob) * np.array([[0, 1],
                                                [0, 0]])

    residual_kraus = np.diag([1, np.sqrt(1-damp_prob)])
    return [residual_kraus, damping_op]

def append_kraus_to_gate(kraus_ops, g):
    """
    Follow a gate `g` by a Kraus map described by `kraus_ops`.

    :param list kraus_ops: The Kraus operators.
    :param numpy.ndarray g: The unitary gate.
    :return: A list of transformed Kraus operators.
    """
    return [kj.dot(g) for kj in kraus_ops]

def append_damping_to_gate(gate, damp_prob=.1):
    """
    Generate the Kraus operators corresponding to a given unitary
    single qubit gate followed by an amplitude damping noise channel.

    :params np.ndarray|list gate: The 2x2 unitary gate matrix.
    :params float damp_prob: The one-step damping probability.
    :return: A list [k1, k2] of the Kraus operators that parametrize the map.
    :rtype: list
    """
    return append_kraus_to_gate(damping_channel(damp_prob), gate)
```

```python
%%time

# single step damping probability
damping_per_I = 0.02

# number of program executions
trials = 200

results_damping = []
lengths = np.arange(0, 201, 10, dtype=int)
for jj, num_I in enumerate(lengths):

    print("\r{}/{}, ".format(jj, len(lengths)), end="")

    p = Program(X(0))
    ro = p.declare("ro")
    # want increasing number of I-gates
    p.inst([I(0) for _ in range(num_I)])
    p.inst(MEASURE(0, ro[0]))

    # overload identity I on qc 0
    p.define_noisy_gate("I", [0], append_damping_to_gate(np.eye(2), damping_per_I))
    p.wrap_in_numshots_loop(trials)
    qc.qam.random_seed = int(num_I)
    res = qc.run(p)
    results_damping.append([np.mean(res), np.std(res) / np.sqrt(trials)])

results_damping = np.array(results_damping)
```

    20/21, CPU times: user 74.2 ms, sys: 6.16 ms, total: 80.3 ms
    Wall time: 744 ms

```python
dense_lengths = np.arange(0, lengths.max()+1, .2)
survival_probs = (1-damping_per_I)**dense_lengths
logpmf = binom.logpmf(np.arange(trials+1)[np.newaxis, :], trials, survival_probs[:, np.newaxis])/np.log(10)
```

```python
DARK_TEAL = '#48737F'
FUSCHIA = "#D6619E"
BEIGE = '#EAE8C6'
cm = colors.LinearSegmentedColormap.from_list('anglemap', ["white", FUSCHIA, BEIGE], N=256, gamma=1.5)
```

```python
plt.figure(figsize=(14, 6))
plt.pcolor(dense_lengths, np.arange(trials+1)/trials, logpmf.T, cmap=cm, vmin=-4, vmax=logpmf.max())
plt.plot(dense_lengths, survival_probs, c=BEIGE, label="Expected mean")
plt.errorbar(lengths, results_damping[:,0], yerr=2*results_damping[:,1], c=DARK_TEAL,
             label=r"noisy qvm, errorbars $ = \pm 2\hat{\sigma}$", marker="o")
cb = plt.colorbar()
cb.set_label(r"$\log_{10} \mathrm{Pr}(n_1; n_{\rm trials}, p_{\rm survival}(t))$", size=20)

plt.title("Amplitude damping model of a single qubit", size=20)
plt.xlabel(r"Time $t$ [arb. units]", size=14)
plt.ylabel(r"$n_1/n_{\rm trials}$", size=14)
plt.legend(loc="best", fontsize=18)
plt.xlim(*lengths[[0, -1]])
plt.ylim(0, 1)
```

# Example 2: dephased CZ-gate

Dephasing is usually characterized through a qubit's $T_2$ time.
For a single qubit the dephasing Kraus operators are
$$
K_1(p) = \sqrt{1-p} I_2 \\
K_2(p) = \sqrt{p} \sigma_Z
$$
where $p = 1 - \exp(-T_2/T_{\rm gate})$ is the probability that the qubit is dephased over the time interval of interest, $I_2$ is the $2\times 2$-identity matrix and $\sigma_Z$ is the Pauli-Z operator.

For two qubits, we must construct a Kraus map that has *four* different outcomes:

1. No dephasing
2. Qubit 1 dephases
3. Qubit 2 dephases
4. Both dephase

The Kraus operators for this are given by
\begin{align}
K'_1(p,q) = K_1(p)\otimes K_1(q) \\
K'_2(p,q) = K_2(p)\otimes K_1(q) \\
K'_3(p,q) = K_1(p)\otimes K_2(q) \\
K'_4(p,q) = K_2(p)\otimes K_2(q)
\end{align}
where we assumed a dephasing probability $p$ for the first qubit and $q$ for the second.

Dephasing is a *diagonal* error channel and the CZ gate is also diagonal, therefore we can get the combined map of dephasing and the CZ gate simply by composing $U_{\rm CZ}$ the unitary representation of CZ with each Kraus operator
\begin{align}
K^{\rm CZ}_1(p,q) = K_1(p)\otimes K_1(q)U_{\rm CZ} \\
K^{\rm CZ}_2(p,q) = K_2(p)\otimes K_1(q)U_{\rm CZ} \\
K^{\rm CZ}_3(p,q) = K_1(p)\otimes K_2(q)U_{\rm CZ} \\
K^{\rm CZ}_4(p,q) = K_2(p)\otimes K_2(q)U_{\rm CZ}
\end{align}

**Note that this is not always accurate, because a CZ gate is often achieved through non-diagonal interaction Hamiltonians! However, for sufficiently small dephasing probabilities it should always provide a good starting point.**

```python
def dephasing_kraus_map(p=.1):
    """
    Generate the Kraus operators corresponding to a dephasing channel.

    :params float p: The one-step dephasing probability.
    :return: A list [k1, k2] of the Kraus operators that parametrize the map.
    :rtype: list
    """
    return [np.sqrt(1-p)*np.eye(2), np.sqrt(p)*np.diag([1, -1])]

def tensor_kraus_maps(k1, k2):
    """
    Generate the Kraus map corresponding to the composition
    of two maps on different qubits.

    :param list k1: The Kraus operators for the first qubit.
    :param list k2: The Kraus operators for the second qubit.
    :return: A list of tensored Kraus operators.
    """
    return [np.kron(k1j, k2l) for k1j in k1 for k2l in k2]

```

```python
%%time
# single step damping probabilities
ps = np.linspace(.001, .5, 200)

# number of program executions
trials = 500

results = []

for jj, p in enumerate(ps):

    corrupted_CZ = append_kraus_to_gate(
    tensor_kraus_maps(
        dephasing_kraus_map(p),
        dephasing_kraus_map(p)
    ),
    np.diag([1, 1, 1, -1]))

    print("\r{}/{}, ".format(jj, len(ps)), end="")

    # make Bell-state
    p = Program(H(0), H(1), CZ(0,1), H(1))
    ro = p.declare("ro", memory_size=2)
    p.inst(MEASURE(0, ro[0]))
    p.inst(MEASURE(1, ro[1]))

    # overload identity I on qc 0
    p.define_noisy_gate("CZ", [0, 1], corrupted_CZ)
    p.wrap_in_numshots_loop(trials)
    qc.qam.random_seed = jj
    res = qc.run(p)
    results.append(res)

results = np.array(results)
```

    199/200, CPU times: user 568 ms, sys: 43.2 ms, total: 611 ms
    Wall time: 1.89 s

```python
Z1s = (2*results[:,:,0]-1.)
Z2s = (2*results[:,:,1]-1.)
Z1Z2s = Z1s * Z2s

Z1m = np.mean(Z1s, axis=1)
Z2m = np.mean(Z2s, axis=1)
Z1Z2m = np.mean(Z1Z2s, axis=1)
```

```python
plt.figure(figsize=(14, 6))
plt.axhline(y=1.0, color=FUSCHIA, alpha=.5, label="Bell state")

plt.plot(ps, Z1Z2m, "x", c=FUSCHIA, label=r"$\overline{Z_1 Z_2}$")
plt.plot(ps, 1-2*ps, "--", c=FUSCHIA, label=r"$\langle Z_1 Z_2\rangle_{\rm theory}$")

plt.plot(ps, Z1m, "o", c=DARK_TEAL, label=r"$\overline{Z}_1$")
plt.plot(ps, 0*ps, "--", c=DARK_TEAL, label=r"$\langle Z_1\rangle_{\rm theory}$")

plt.plot(ps, Z2m, "d", c="k", label=r"$\overline{Z}_2$")
plt.plot(ps, 0*ps, "--", c="k", label=r"$\langle Z_2\rangle_{\rm theory}$")

plt.xlabel(r"Dephasing probability $p$", size=18)
plt.ylabel(r"$Z$-moment", size=18)
plt.title(r"$Z$-moments for a Bell-state prepared with dephased CZ", size=18)
plt.xlim(0, .5)
plt.legend(fontsize=18)
```


# Hello World

It is not really established yet, what the quantum analogy of the classical `print("Hello world")` is. I will define here the "Hello world" as the task to build an arbitrary circuit with rotations and an entangling gate on 2 qubits, I will print the circuit and sample from it.

## Tensorflow Quantum "Hello World"

Since Tensorflow Quantum (tfq) does all the quantum part on google's own quantum platform Cirq, the "Hello world" example can be done without using tfq.

```python
import tensorflow as tf
import tensorflow_quantum as tfq

import cirq
import sympy                            # Sympy is for symbolic math
import numpy as np

from cirq.contrib.svg import SVGCircuit # to print the circuit
```

- Define symbolic variables, which will be the parameters of the rotations
- Define qubits q0 and q1
- Define a circuit with a Rx rotations on each qubit followed by CNOT
- Print the circuit with the Cirquit native `SVGCircuit`

```python
x1, x2 = sympy.symbols('x1 x2')             # Define symbolic parameters x1, x2

q0, q1 = cirq.GridQubit.rect(1,2)          # Define two qubits on grid

# Define a circuit
tf_circuit = cirq.Circuit(
    cirq.rx(x1).on(q0),
    cirq.rx(x2).on(q1),
    cirq.CNOT(control=q0, target=q1)
)

SVGCircuit(tf_circuit)                         # Print circuit
```

- Or simply print the circuit

```python
print(tf_circuit)
```

    (0, 0): ───Rx(x1)───@───
                        │
    (0, 1): ───Rx(x2)───X───

### How to measure an output?

In the cirq simulator we can get the state vector as an output and after that we can calculate any observable that we want.

#### Define the Cirq simulator and the resolver

The resolver sets the symbolic values of the rotation angles to a real value.

```python
simulator = cirq.Simulator()
resolver = cirq.ParamResolver({x1: 0.5, x2: -0.5}) # Set symbolic parameters to values 0.5 and -0.5
```

#### Do a measurement

Measurements in Cirq can be by adding `cirq.measure()` and define which qubit we want to measure. The "key" in the measurement can be set arbitrary and is just to find the histogram later. Cirq measurements can be done the `.run` command from the `cirq.Simulator()`.

```python
resolved_circuit = cirq.resolve_parameters(tf_circuit, resolver) # Resolve sympy values in circuit
resolved_circuit.append(cirq.measure(q0, key='some_key'))            # Add a measurement in Z direction on qubit 0
results = simulator.run(resolved_circuit, repetitions=100)    # Run a simulation of this circuit 100 times
print(results.histogram(key='some_key'))                             # Get measurement statistics
```

    Counter({0: 90, 1: 10})

#### Get measurement by simulating the state vector

Instead of running the measurement 100 times we can also get analytic results from the `.simulate` command from the `cirq.Simulator()`.
The measurement direction can here be specified as an arbitrary combination of Pauli operators.

```python
output_state_vector = simulator.simulate(tf_circuit, resolver).final_state # simulate state vector

z0 = cirq.Z(q0)              # Define measurement direction in Z direction

qubit_map = {q0: 0, q1: 1}   # Define which qubits are involved in measurement

z0.expectation_from_wavefunction(output_state_vector, qubit_map).real
```

    0.8775825500488281

Here is another example for a measurement in $0.5 Z_0 + X_1$ direction

```python
z0x1 = 0.5 * z0 + cirq.X(q1)

z0x1.expectation_from_wavefunction(output_state_vector, qubit_map).real
```

    0.43879127502441406

## Pennylane "Hello World"

Pennylane is "hardware agnostic", which means we do not have to stick to one quantum environment like Cirq. For every environment Qiskit, Cirq, Forest we basically just have to change one line of code, the `qml.device()`.

```python
import pennylane as qml
from pennylane import numpy as np
```

### Define a device

This is basically the whole magic of pennylane. By defining a device we can access all possible quantum hardware platforms and simulators. For more information see [here](https://pennylane.ai/plugins.html).

We will start with the `qiskit.aer` device that can be installed via `pip install pennylane-qiskit`.

```python
dev1 = qml.device("qiskit.aer", wires=2)
```

```python
def circuit(params):
    qml.RX(params[0], wires=0)
    qml.RX(params[1], wires=1)
    qml.CNOT(wires=[0,1])
    return qml.expval(qml.PauliZ(0))
```

Pennylane allows us to use wrappers, to directly define a `circuit()` for a certain device `dev1`. To do so you can start the above definition of the circuit with `@qml.qnode(dev)`. This will look the following way:

```
@qml.qnode(dev1)
def circuit(params):
    qml.RX(params[0], wires=0)
    qml.RY(params[1], wires=0)
    return qml.expval(qml.PauliZ(0))
```

I recommend doing this, if you anyway stick to the same device. I will not do it in this tutorial, because I will change the device many times.

To set the device for the circuit we can also do the following and evaulate it directly for the given parameters

```python
q_circuit = qml.QNode(circuit, dev1)

params = np.array([0.5, -0.5])
q_circuit(params)
```

    0.861328125

To print the circuit with `qiskit.aer` we can directly do the following:

```python
dev1._circuit.draw()
```

<pre style="word-wrap: normal;white-space: pre;background: #fff0;line-height: 1.1;font-family: "Courier New",Courier,monospace">        ┌─────────┐      ┌─┐
q_0: |0>┤ Rx(0.5) ├───■──┤M├───
        ├─────────┴┐┌─┴─┐└╥┘┌─┐
q_1: |0>┤ Rx(-0.5) ├┤ X ├─╫─┤M├
        └──────────┘└───┘ ║ └╥┘
 c_0: 0 ══════════════════╩══╬═
                             ║
 c_1: 0 ═════════════════════╩═
                               </pre>

With the `default.qubit` the output looks the following way.

```python
dev1 = qml.device("default.qubit", wires=2)
q_circuit = qml.QNode(circuit, dev1)
params = np.array([0.5, -0.5])
q_circuit(params)
print(q_circuit.draw())
```

     0: ──RX(0.5)───╭C──┤ ⟨Z⟩
     1: ──RX(-0.5)──╰X──┤

We can evaluate the circuit like before with the parameters 0.5 and -0.5

```python
params = np.array([0.5, -0.5])
q_circuit(params)
```

    0.8775825618903726

And we can also sample the measurement outputs by changing the return line of the circuit function from `qml.expval()` to `qml.sample()`

```python
def circuit_sample(params):
    qml.RX(params[0], wires=0)
    qml.RX(params[1], wires=1)
    qml.CNOT(wires=[0,1])
    return qml.sample(qml.PauliZ(0)), qml.sample(qml.PauliZ(1))

dev2 = qml.device("default.qubit", wires=2)
q_circuit = qml.QNode(circuit_sample, dev2)
params = np.array([0.5, -0.5])
q_circuit(params)
```

    array([[ 1,  1,  1, ...,  1,  1,  1],
           [ 1, -1,  1, ...,  1,  1,  1]])

And the wavefunction can as well be accessed with the `default.qubit`

```python
q_circuit(params)
print(dev1.state)
```

    [0.93879128+0.j         0.        +0.23971277j 0.06120872+0.j
     0.        -0.23971277j]

To change the measurement of the circuit, we have to redefine the circuit. It seems there is no way around this. We can change the last line of the `def circuit_sample` function to e.g. `return qml.expval(qml.PauliZ(0) @ qml.PauliX(1))` to measure the correlation $\langle Z \otimes X \rangle$.


SNe parametric model (Villar et al, ALeRCE):

\begin{align}
early(t) &=\frac{A\left(1-f\frac{t-t_0}{\gamma}\right)}{1+exp\left(\frac{-(t-t_0)}{t_{rise}}\right)}\left(1-\sigma\left(\frac{t-(\gamma + t_0)}{3}\right)\right)\\
late(t) &= \frac{A(1-f)exp\left(\frac{-(t-(\gamma+t_0))}{t_{fall}}\right)}{1+exp\left(\frac{-(t-t_0)}{t_{rise}}\right)}\sigma\left(\frac{t-(\gamma + t_0)}{3}\right)\\
flux(t) &= early(t)+late(t)>0
\end{align}
Where $t$ is the time, in day units, of the light curve, and $\sigma(t)=1/(1+exp(-t))$ is the sigmoid function.

Parameters:
1. $t_0$ time close to the maximum curve flux (peak).
1. $A>0$ controls the global curve scale.
1. $\gamma>0$ associated with the plateau duration.
1. $t_{rise}>0$ controls the time to maximum flux value (peak).
1. $t_{fall}>0$ mostly associated with the flux fall duration after plateau.
1. $0<f<1$ associated with plateau slope value.

```python
import sys
sys.path.append('../../../')
sys.path.append('../../../../fuzzy-tools')
```

```python
%load_ext autoreload
%autoreload 2
import matplotlib.pyplot as plt
from lchandler.synthetic.synthetic_curve_generators import syn_sne_fnumpy
import fuzzytools.cuteplots.colors as cc
from matplotlib import cm
import numpy as np
import random

NT = 400
N = 10
times = np.linspace(-10, 200, NT)

A = 1
t0 = 80
gamma = 40
f = 0.5
trise = 8
tfall = 10

fig, axs = plt.subplots(2, 2, figsize=(10, 8))
axs = [axs[0,0], axs[1,0],axs[0,1],axs[1,1]]
ch_list = []
ch_list.append(zip(np.linspace(-100, 100, N),['gamma']*N,['$\gamma$']*N))
ch_list.append(zip(np.linspace(0, 1, N),['f']*N,['f']*N))
ch_list.append(zip(np.linspace(1, 20, N),['trise']*N,['$t_{rise}$']*N))
ch_list.append(zip(np.linspace(5, 20, N),['tfall']*N,['$t_{fall}$']*N))

for k,ax in enumerate(axs):
    cmap = cm.get_cmap('viridis', N)
    colors = cmap.colors
    for kv,(v_value,v_name,v_legend) in enumerate(ch_list[k]):
        kwargs = {
            'A':A,
            't0':t0,
            'gamma':gamma,
            'f':f,
            'trise':trise,
            'tfall':tfall,
        }
        kwargs[v_name] = v_value
        flux = syn_sne_fnumpy(times, *[kwargs[k] for k in kwargs.keys()])
        label = f'{v_legend}={v_value:.3f}'
        ax.plot(times, flux, c=colors[kv], label=label)
        ax.axvline(t0, c='k', label='t0' if kv==0 else None, alpha=1, lw=1)
        ax.axvline(t0+gamma, c='k', label='t0' if kv==0 else None, alpha=1, lw=1)

    ax.set_title(f'different curves by changing only {v_legend}')
    ax.legend()
    if k in [1,3]:
        ax.set_xlabel('days')
    #ax.set_ylabel('flux')
    ax.grid(alpha=0.5)

fig.tight_layout()
plt.show()
```

```python
%load_ext autoreload
%autoreload 2
import matplotlib.pyplot as plt
from fuzzytools.cutePlots.plots import plot_hist_bins
import fuzzytools.cutePlots.colors as cpc
from scipy.stats import gamma

sigma_error_arguments = {
    'loc':1e-3,
    'shape':1.8,
    'scale':0.01,
}
N = int(1e4)
shape, scale = sigma_error_arguments['shape'], sigma_error_arguments['scale']
samples = np.random.gamma(shape, scale, size=N)
samples += sigma_error_arguments['loc']

data_dic = {'gamma distribution':samples}
fig, ax, handles = plot_hist_bins(data_dic, uses_density=True, return_legend_patches=True)
x = np.linspace(0, np.max(samples), int(1e4))
y = gamma.pdf(x, shape, sigma_error_arguments['loc'], scale)
label = f'gamma(shape={shape}, scale={scale})'
ax.plot(x, y, linewidth=2, color=cpc.colors()[1], label=label)
handles_, labels = ax.get_legend_handles_labels()
ax.legend(handles=handles+handles_)
plt.plot()
```

```python
%load_ext autoreload
%autoreload 2
import numpy as np
import random
from src.synth.synthetic_SNe import generate_FSNe
from itertools import chain
from fuzzytools.myUtils.progress_bars import ProgressBar
import astrolightcurveshandler.src.data_classes as lcdc # auxiliar import
from src import C_

survey_name = 'FakeSNe'
band_names = ['g', 'r']
class_names = ['short', 'long','small']
description = 'fake supernovaes for sanity checks'

# PREPARE NEW DATASET
lcset = lcdc.LCSet(
    {},
    survey_name,
    description,
    band_names,
    class_names,
    True,
)
raw_lcdataset = lcdc.LCDataset()
raw_lcdataset.set_raw(lcset)

N_curvesteps = 20
times = np.linspace(-100, 100, NT)
max_nc = 1e3
nc_dict = {
    'short':.6,
    'long':.3,
    'small':.1,
}
curve_points_dropout = 0.92

labels = list(chain.from_iterable([[kc]*int(nc_dict[c]*max_nc) for kc,c in enumerate(class_names)]))
random.shuffle(labels)
print(f'labels: {len(labels)}')

bar = ProgressBar(len(labels))
for k,label in enumerate(labels):
    obj_name = k
    bar(f'obj_name: {obj_name}')
    lcobj = lcdc.LCO()
    SNe_kwargs = None
    for kb,b in enumerate(band_names):
        SNe_type = class_names[label]
        f_return = generate_FSNe(SNe_type, times, curve_points_dropout, sigma_error_arguments, SNe_kwargs=SNe_kwargs)
        band_lc_flux, theorical_model = f_return
        lcobj.add_b(b, band_lc_flux[:,0], band_lc_flux[:,1], band_lc_flux[:,2])
        SNe_kwargs = theorical_model.copy()
        SNe_kwargs['A'] *= 0.9 # example for slightly different bands

    lcobj.reset_day_offset_serial()
    lcobj.set_y(label)
    lcobj.z = 0 # redshift
    lcobj.ra = 0
    lcobj.dec = 0
    raw_lcdataset.raw.data[obj_name] = lcobj

bar.done()
```

```python
import fuzzytools.myUtils.lists as lists
import matplotlib.pyplot as plt
from src.plots import plot_lightcurve

lcobj, key = raw_lcdataset.raw.get_random_lcobj()
#print(lcobj)
fig, ax = plt.subplots(1,1)
for kb,b in enumerate(raw_lcdataset.raw.band_names):
    plot_lightcurve(ax, lcobj, b)
```

```python
%load_ext autoreload
%autoreload 2
from fuzzytools.myUtils.files import save_pickle
from src import C_

save_folder = '../save/{}'.format(survey_name)
filename = f'survey-{survey_name}_bands-{"".join(band_names)}_classes-{len(class_names)}'
filedir = f'{save_folder}/{filename}.{C_.EXT_RAW_LIGHTCURVE}'
save_pickle(filedir, raw_lcdataset)
```

```python
import random
import numpy as np
import matplotlib.pyplot as plt
import fuzzytools.cutePlots.colors as cpc
from src import C_

def get_random_keys(data, class_names, Nc):
    all_keys = list(data.keys())
    random.shuffle(all_keys)
    counter = {c:0 for c in class_names}
    keys = []
    for key in all_keys:
        y = data[key]['y']
        if counter[class_names[y]] < Nc:
            keys.append(key)
            counter[class_names[y]] += 1
    return keys

figsize = (12,10)
fig, axs = plt.subplots(len(band_names), 1, figsize=figsize)

Nc = 1
keys = get_random_keys(data, class_names, Nc)
print('keys:',keys)

#colors = cm.Dark2(np.linspace(.0, .999, len(keys)))
colors = cpc.colors()[0:]
for kb,band in enumerate(band_names):
    ax = axs[kb]
    for k,key in enumerate(keys):
        obj = data[key]
        class_name = class_names[obj['y']]
        lc = obj['x_mb'][band]
        iday = obj['iday']
        print(iday)
        days = lc[:,0]
        c = colors[k]
        plot_lightcurve(ax, days, lc[:,1], lc[:,2], label=f'{class_name}', color=c)

    ax.set_title(f'survey_name: {survey_name} - band: {band}')
    ax.legend()
    ax.grid(alpha=0.5)
    #ax.invert_yaxis()
    ax.set_ylabel('flux')

ax.set_xlabel('days')
plt.plot()
```

```python
%matplotlib inline
import seaborn as sns
import numpy as np
import pandas as pd

samples = []
band = 'g'
for k,key in enumerate(data.keys()):
    parameters = data[key]['tm_mb'][band]
    parameters = [parameters[k] for k in parameters][:-2]
    samples.append(np.array(parameters)[None])

samples = np.concatenate(samples, axis=0)
labels = ['A','t0','$\gamma$','f','$t_{rise}$','$t_{fall}$']
df = pd.DataFrame(samples, columns=labels)
df['class'] = [class_names[data[key]['y']] for key in data.keys()]
print(df)
figsize = (5, 5)
sns.pairplot(df, hue='class')
```

```python
import corner
import numpy as np

keys = data.keys()
samples = []
band = 'g'
for k,key in enumerate(keys):
    parameters = data[key]['tm_mb'][band]
    parameters = [parameters[k] for k in parameters]
    sample = np.array(parameters)[None,:-2]
    samples.append(sample)

figsize = (3, 3)
samples = np.concatenate(samples, axis=0)
labels = ['A','t0','$\gamma$','f','$t_{rise}$','$t_{fall}$']
figure = corner.corner(samples, labels=labels, figsize=figsize)
```


## Bayesian Neural Networks Background with code snippets

BNNs represent the integration of a hierarchical Bayesian framework together with a neural network structure composed of recursive applications of linear weighted functions followed by nonlinear transformations. With prior distributions on weights, we are able to approximate their posterior distributions and perform posterior prediction via a [variational inference](https://en.wikipedia.org/wiki/Variational_Bayesian_methods) framework.
Consider a model that returns a probability distribution over an output $y$ given an input $x$ and parameter $\theta$, that is $p(y|x,\theta)$. The goal is to learn these parameters from the observed data $D = \{D_i\}_{i=1}^N = \{x_i,y_i\}_{i=1}^N$. Following a Bayesian approach, we put a prior distribution $p(\theta)$ over $\theta$ and aim to obtain the posterior distribution

$$p(\theta|D)\propto p(\theta) p(D|\theta) = p(\theta) \prod_{i=1}^N p(y_i|\theta, x_i). $$

In most cases, the posterior distribution is intractable and an approximation is required. Consider using a variational family distribution $q_v(\theta)$ with parameters $v$ to approximate the posterior by minimizing the KL divergence, i.e.,

\begin{equation}\label{eq:minKL}
\min_v D_{KL}{q_v(\theta)}{p(\theta|D)}
\end{equation}

where,
$$
D_{KL}{q_v(\theta)}{p(\theta|D)} = \mathbb{E}_{q} \bigg[ {\log\frac{q_v(\theta)}{p(\theta|D)}} \bigg] \\
$$

$$
= \mathbb{E}_{q} \bigg[ {\log\frac{q_v(\theta) p(D)}{p(D|\theta)p(\theta)}} \bigg]
$$

$$
= - \left\{\mathbb{E}_{q}{ [\log p(D|\theta) ]}- D_{KL}{q_v(\theta)}{p(\theta)}\right\} + \log p(D)
$$

where the first term represents the log-likelihood of the model predictions, while the second part serves as a regularizer KL divergence between the approximate posterior $q_v(\theta)$ and the prior distribution $p(\theta)$. In the case where $\log p(D)$ is constant given prior distribution $p(\theta)$, minimizing the KL divergence is equivalent to maximizing the objective function or the Evidence Lower BOund (ELBO) given by
\begin{equation}
L(v) = \sum_{i=1}^N \mathbb{E}_{q}{\log p(y_i|\theta,x_i)} - \beta D_{KL}{q_v}{p}, \label{eq:Lv}
\end{equation}

```python

#############################################################################
##
## CNN_BNN_Model.py
##
#############################################################################

# %% Loss
# Convert the target to a one-hot tensor of shape (batch_size, 10) and
# with a on-value of 1 for each one-hot vector of length 10.
labels_distribution = tfd.Categorical(logits=logits,name='label_dist')

# Compute the -ELBO as the loss, averaged over the batch size.
neg_log_likelihood = -tf.reduce_mean(input_tensor=labels_distribution.log_prob(label))

kl = sum(model.losses) / N

KLscale=1.0
Loss_ = neg_log_likelihood + KLscale * kl

```

```python
#############################################################################
##
## CIFAR10_BNN.py
##
#############################################################################

kl_regularizer = FLAGS.kl_regularizer

# Compute the -ELBO as the loss. The kl term is annealed from 0 to 1 over
# the epochs specified by the kl_annealing flag.
log_likelihood = labels_distribution.log_prob(labels)
neg_log_likelihood = -tf.reduce_mean(input_tensor=log_likelihood)
kl = sum(model.losses) / len(x_train) * tf.minimum(1.0, kl_regularizer)

loss = neg_log_likelihood + kl

```

- [losses](https://www.tensorflow.org/probability/api_docs/python/tfp/layers/Convolution2DFlipout)

  + Upon being built, this layer adds losses (accessible via the losses property) representing the divergences of kernel and/or bias surrogate posteriors and their respective priors. When doing minibatch stochastic optimization, make sure to scale this loss such that it is applied just once per epoch (e.g. if kl is the sum of losses for each element of the batch, you should pass kl / num_examples_per_epoch to your optimizer).

The $\beta$ is a hyperparameter for tuning the degree of regularization during training of the neural network. Typically the $\beta$ hyperparameter is introduced to address the  difficulty of the {KL-term vanishing} which is often observed while training Variational Auto Encoders (VAEs). This hyperparameter was introduced by Bowman et.al [1] as *KL-annealing* where the $\beta$ parameter can be varied from 0 to 1 over the course of training.

Recently an analysis on the scheduling strategies of $\beta$ was presented by Liu et.al [2]. Once the BNN model is trained the inference procedure is to compute the predictive probability distribution $p(y^{*}|x^{*})$ given by

\begin{equation}
p(y^{*}|x^{*}) = \int p_{w}(y^{*}|x^{*})p(w) dw \, ,
\label{Eq:InferPost}
\end{equation}

where $x^{*}$ is the unseen (test) sample and $y^{*}$ is the corresponding predicted class.
Finding a closed form solution to the above integral for non conjugate pair of distribution is not possible hence the integral can be approximated as an expectation by sampling from $q_v(w|D)$ as

\begin{equation}
\mathbb{E}_{q}{p(y^{*}|x^{*})} = \int q_v(\theta | D) p_{w}(y|x) dw \approx \frac{1}{S} \Sigma_{i=1}^{S} p_{w_i}(y^{*}|x^{*})
\end{equation}

where $S$ are the number of samples or Monte-Carlo iterations.

# Training Procedure

- The procedure is summarize to optimize the loss function described in minimization of KL divergence. The backpropagation algorithm is at the heart of training a neural network and  relies of the calculation of gradients of the loss function which are subsequently used to update the weights layer by layer. There are several approaches available for computing the gradients such as  score-function gradient estimators shown in [3,4] reparameterization  gradient  estimators  described in [5,6], or a combination of the two as described in Naesseth et.al [7]. These methods provide an unbiased stochastic gradient which is used to calculate the optimum for the loss function.

- In training the model reparametrization procedure for computing the gradients is used.
We use the Gaussian distribution for the variational posterior and initilize the weights, $\theta$, by sampling from the unit Gaussian with mean $\mu$ and standard-deviation $\sigma$.

- The hyperparameters for the variational posterior are therefore $v = (\mu, \sigma)$. The reparametization trick is to parameterize weights as a function, $t$, where $\theta = t(v, \epsilon) = \mu + {\rm{log}}(1 + exp(\sigma)) \circ \epsilon $, where $\circ$ represents an element-wise multiplication and the parameter free noise is defined by $\epsilon \sim \mathit{N}(0, I)$. Eq. for Loss $L$ defined earlier can be rewritten in terms of $L(\theta,v)$ and the gradients, $\Delta_{\mu} L(\theta,v)$ and $\Delta_{\sigma} L(\theta,v)$, are computed to update the hyperparamters $ \mu  \leftarrow  \mu + \eta \Delta_{\mu} L(\theta,v) $ and  $ \sigma  \leftarrow  \sigma + \eta \Delta_{\sigma} L(\theta,v) $.

# Distributed Training Using Horovod

+ Setting the distributed optimizer,with scaling the learning rare with horovod size.

```python
##############
#
# Demo Code
#
###############
# Horovod: adjust learning rate based on number of GPUs.
opt = tf.train.AdamOptimizer(FLAGS.learning_rate * hvd.size())

# Horovod: add Horovod Distributed Optimizer.
opt = hvd.DistributedOptimizer(opt)

```

+ Hooks to setup the monitored training session.

```python
##############
#
# Demo Code
#
###############

hooks = [
    # Horovod: BroadcastGlobalVariablesHook broadcasts initial variable states
    # from rank 0 to all other processes. This is necessary to ensure consistent
    # initialization of all workers when training is started with random weights
    # or restored from a checkpoint.
    hvd.BroadcastGlobalVariablesHook(0),

    # Horovod: adjust number of steps based on number of GPUs.
    tf.train.StopAtStepHook(last_step= Num_iter // hvd.size()),

    tf.train.LoggingTensorHook(tensors={'step': global_step, 'loss': Loss_},every_n_iter=500),
]
```

+ Config for horovod, more details can be seen in horovod tutorials.

```python

##############
#
# Demo Code
#
###############
# # Horovod: pin GPU to be used to process local rank (one GPU per process)
config.gpu_options.allow_growth = True
config.gpu_options.visible_device_list = str(hvd.local_rank())

checkpoint_dir = os.path.join(dirmake,'checkpoints') if hvd.rank() == 0 else None

```

+ Monitored training session to take cares of initialization and hooks to create a session to run various operations in the tensorflow graph etc. The while loop contains the training code.

```python
##############
#
# Demo Code
#
###############

# The MonitoredTrainingSession takes care of session initialization,
# restoring from a checkpoint, saving to a checkpoint, and closing when done
# or an error occurs.
with tf.train.MonitoredTrainingSession(checkpoint_dir=checkpoint_dir,hooks=hooks,config=config) as mon_sess:
    #Train Time start
    start_Train_time = time.time()
    iter_num = 0

    while not mon_sess.should_stop():
        # Training code

```

+ **EXAMPLE BNN Network Summary**

```
Model: CIFAR-10 Bayesian Model "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
Conv_I (Conv2DFlipout)       (None, 32, 32, 32)        1760
_________________________________________________________________
Conv_II (Conv2DFlipout)      (None, 32, 32, 32)        18464
_________________________________________________________________
Max_I_ (MaxPooling2D)        (None, 16, 16, 32)        0
_________________________________________________________________
Conv_III (Conv2DFlipout)     (None, 16, 16, 64)        36928
_________________________________________________________________
Conv_IV (Conv2DFlipout)      (None, 16, 16, 64)        73792
_________________________________________________________________
Max_II (MaxPooling2D)        (None, 8, 8, 64)          0
_________________________________________________________________
Conv_V (Conv2DFlipout)       (None, 8, 8, 128)         147584
_________________________________________________________________
Conv_VI (Conv2DFlipout)      (None, 8, 8, 128)         295040
_________________________________________________________________
Max_III (MaxPooling2D)       (None, 4, 4, 128)         0
_________________________________________________________________
flatten (Flatten)            (None, 2048)              0
_________________________________________________________________
Dense_I (DenseFlipout)       (None, 128)               524416
_________________________________________________________________
Dense_II (DenseFlipout)      (None, 10)                2570
=================================================================
Total params: 1,100,554
Trainable params: 1,100,554
Non-trainable params: 0
_________________________________________________________________
```

# References

[1] Samuel R Bowman, Luke Vilnis, Oriol Vinyals, Andrew M Dai, Rafal Jozefowicz, and SamyBengio. Generating sentences from a continuous space.arXiv preprint arXiv:1511.06349,2015.

[2]  Xiaodong Liu, Jianfeng Gao, Asli Celikyilmaz, Lawrence Carin, et al.  Cyclical annealingschedule: A simple approach to mitigating kl vanishing.arXiv preprint arXiv:1903.10145,2019.

[3] John Paisley, David Blei, and Michael Jordan. Variational bayesian inference with stochas-tic search.arXiv preprint arXiv:1206.6430, 2012.

[4] Rajesh  Ranganath,  Sean  Gerrish,  and  David  M  Blei.   Black  box  variational  inference.arXiv preprint arXiv:1401.0118, 2013.

[5] Charles  Blundell,  Julien  Cornebise,  Koray  Kavukcuoglu,  and  Daan  Wierstra.   Weightuncertainty in neural networks.arXiv preprint arXiv:1505.05424, 2015

[6] Diederik P Kingma and Max Welling. Auto-Encoding Variational Bayes. Technical report

[7] CA  Naesseth,  FJR  Ruiz,  SW  Linderman,  and  DM  Blei.   Reparameterization  gradientsthrough  acceptance-rejection  sampling  algorithms.   InProceedings  of  the  20th  Interna-tional Conference on Artificial Intelligence and Statistics, AISTATS 2017, 2017.

**Author: Himanshu Sharma, Argonne National Laboratory**

```python

```


Homework 6

Yutong Dai yutongd3@illinois.edu

## 6.5
**a)**
$$
\begin{align}
& \min \quad 19 w_1 + 57 w_2\\
& s.t \quad w_1 + 2w_2 \geq 10\\
& \qquad w_1 + 4w_2 \geq 24\\
& \qquad 2w_1 + 3w_2 \geq 20\\
& \qquad 3w_1 + 2w_2 \geq 20\\
& \qquad 5w_1 + 3w_2 \geq 25\\
& \qquad w_1,w_2\geq 0
\end{align}
$$

It's easy to verify $w^*=(w_1^*,w_2^*)=(4, 5)$ is a feasible solution.

**b)** We can treat the dual problem as a primal probelm and the primal problem as it duals. Now we assert that $w^*=(4,5)$ is an optimal solution.
Denote $A$ as the coefficients matrix in **a)**, $b^T=(10,24,20,20,25)$, $c^T=(19,57)$.

By the KKT condition, we want to solve

* ${x{^*}^T}(Aw^* - b) = 0$ $\Rightarrow$ $x_1^*=x_3^*=x_4^*=0$.
* $(c^T - {x^*}^TA)w^* = 0$  $\Rightarrow$ $x_2^*=14, x_5^*=1$.

To summarize, the optimal solution for the primal is $x^*=(0, 14, 0, 0, 1)$ and the optimal solution for the dual is $w^*=(4, 5)$.

## 6.7
**a)**
The dual problem is

$$
\begin{align}
& \min \quad 8 w_1 + 6 w_2\\
& s.t \quad w_1 + w_2 \geq 2\\
& \qquad 2w_1  -2w_2 \geq 3\\
& \qquad 3w_1 + 2w_2 \geq 5\\
& \qquad w_1,w_2\geq 0.
\end{align}
$$

**b)**

* The initial tableau is

|  | $z$ | $x_1$ | $x_2$ | $x_3$ | $x_4$ | $x_5$ | RHS |
| --- | --- | --- | --- | --- | --- | --- | --- |
| $z$ | -1 | 2 | 3 | 5 | 0 | 0 | 0 |
| $x_4$ | 0 | 1 | 2 | 3 | 1 | 0 | 8 |
| $x_5$ | 0 | 1 | -2 | 2 | 0 | 1 | 6 |

then the dual variables take value $(0,0)$.  Then the first 3 constraints are violated.

* After the first iteration the tableau is

|  | $z$ | $x_1$ | $x_2$ | $x_3$ | $x_4$ | $x_5$ | RHS |
| --- | --- | --- | --- | --- | --- | --- | --- |
| $z$ | -1 | 1/3 | -1/3 | 0 | -5/3 | 0 | -40/3 |
| $x_3$ | 0 | 1/3 | 2/3 | 1 | 1/3 | 0 | 8/3 |
| $x_5$ | 0 | 1/3 | -10/3 | 0 | -2/3 | 1 | 2/3 |

then the dual variables take value $(5/3,0)$.  Then the first and second constraints are violated.

* After the second iteration the tableau is

|  | $z$ | $x_1$ | $x_2$ | $x_3$ | $x_4$ | $x_5$ | RHS |
| --- | --- | --- | --- | --- | --- | --- | --- |
| $z$ | -1 | 0 | 3 | 0 | -1 | -1 | -14 |
| $x_3$ | 0 | 0 | 4 | 1 | 1 | -1 | 2 |
| $x_1$ | 0 | 1 | -10 | 0 | -2 | 3 | 2 |

then the dual variables take value $(1,1)$.  Then the second constraint is violated.

* After the third iteration the tableau is

|  | $z$ | $x_1$ | $x_2$ | $x_3$ | $x_4$ | $x_5$ | RHS |
| --- | --- | --- | --- | --- | --- | --- | --- |
| $z$ | -1 | 0 | 0 | -3/4 | -7/4 | -1/4 | -31/2 |
| $x_2$ | 0 | 0 | 1 | 1/4 | 1/4 | -1/4 | 1/2 |
| $x_1$ | 0 | 1 | 0 | 5/2 | 1/2 | -1/2 | 7 |

then the dual variables take value $(7/4,1/4)$.  Since we are at the optimality, no constraint is violated.

**c)**

Rewrite the dual problem in standard form.

$$
\begin{align}
& \min \quad 8 w_1 + 6 w_2\\
& s.t \quad w_1 + w_2 -w_3 = 2\\
& \qquad 2w_1  -2w_2-w_4 = 3\\
& \qquad 3w_1 + 2w_2 -w_5 = 5\\
& \qquad w_1,...,w_5\geq 0.
\end{align}
$$

* At the initial stage:

    * The basic dual variables are $w_3,w_4,w_5$. Nonbasic  variables are $w_1,w_2$.
    * The dual basis is
    $$
    \begin{bmatrix}
        -1 & 0 & -1 \\
        0 & -1 & 0 \\
        0 & 0 & -1
    \end{bmatrix}
    $$

* After the first iteration:

    * The basic dual variables are $w_1,w_3,w_4$. Nonbasic  variables are $w_2,w_5$.
    * The dual basis is
    $$
    \begin{bmatrix}
        1 & -1 & 0 \\
        2 & 0 & -1 \\
        3 & 0 & 0
    \end{bmatrix}
    $$

* After the second iteration:

    * The basic dual variables are $w_1,w_2,w_4$. Nonbasic  variables are $w_3,w_5$.
    * The dual basis is
    $$
    \begin{bmatrix}
        1 & 1 & 0 \\
        2 & -2 & -1 \\
        3 & 2 & 0
    \end{bmatrix}
    $$

* After the third iteration:

    * The basic dual variables are $w_1,w_2,w_5$. Nonbasic  variables are $w_3,w_4$.
    * The dual basis is
    $$
    \begin{bmatrix}
        1 & 1 & 0 \\
        2 & -2 & 0 \\
        3 & 2 & -1
    \end{bmatrix}
    $$

**d)** The objective value for the dual problem follows $0 \rightarrow 40/3 \rightarrow 14 \rightarrow 31/2$, hence keeping worsen.

**e)**

The optimal solution for the primal and dual are $(7,1/2,0)$ and $(7/4, 1/4)$.
It's easy to check that

* Two problems have the same value $31/2$;
* The complementary slackness holding at the optimality.

## 6.8
**a)**

The initial tableau

|  | $z$ | $x_1$ | $x_2$ | $x_3$ | $x_4$ | $x_5$ | RHS |
| --- | --- | --- | --- | --- | --- | --- | --- |
| $z$ | -1 | -6 | 0 | -20 | 0 | 0 | 0 |
| $x_4$ | 0 | 0 | 1 | 4 | 1 | 0 | 10 |
| $x_5$ | 0 | 3 | -1 | 2 | 0 | 1 | 11 |

So the primal problem is

$$
\begin{align}
& \min \quad -6 x_1 - 20 x_3\\
& s.t \quad x_2 + 4x_3 \leq 10\\
& \qquad 3x_1 - x_2 + 2x_3\leq 11\\
& \qquad x_1,x_2,x_3\geq 0.
\end{align}
$$

**b)**
The dual is

$$
\begin{align}
& \max \quad 10w_1 +11 w_2\\
& s.t \quad 3 w_2 \leq -6\\
& \qquad w_1 - w_2 \leq 0\\
& \qquad 4w_1 + 2w_2 \leq -20\\
& \qquad w_1,w_2\leq 0.
\end{align}
$$

**c)** The optimal dual from the tableau is $(-4,-2)$.

## 6.9

**a)** **False**

Consider the primal-dual problem pair,

$$
  \begin{cases}
  & \min\quad x_2 \\
  & s.t \quad -x_1 + x_2 \geq 0 \qquad (P) \\
  & \qquad x_2 \geq -1
  \end{cases}
  \qquad\qquad
  \begin{cases}
  & \max\quad -w_2 \\
  & s.t \quad -w_1 = 0 \qquad (D)\\
  & s.t \quad w_1 + w_2 = -1 \qquad (D)\\
   & \qquad w_1,w_2 \geq 0\\
  \end{cases}
$$

The solution $(x_1,x_2)=(-3,-2)$ is a basic infeasible solution, which gives objective value $-2$ that is smaller than the optimal value 0.
The dual problem is always infeasible.

**b)**

The problem can be rewrite as

$$
\begin{align}
& \min \quad x_1\\
& s.t \quad 2x_1 - x_2 -x_3 = 0\\
& \qquad -2x_1 +3x_2 -x_4 = -6\\
& \qquad x_1,x_2,x_3,x_4\geq 0.
\end{align}
$$

The basic feasible solution for primal is $(0,0,0,6)$. Any basis matrix $B$ leads to dual basic solution $c_B^TB^{−1}$, so we have dual solution $(1/2, 0)$.

This pair of solution is optimal with respect to its problem.

~~Since this is a optimal solution,  the complementary dual are the set $\{(w_1,0)|0\leq w_1  \leq \frac{1}{2}\}$.
The BFS in primal is degenerated while the complementary dual is not unique.~~

**c)** **False**.
Consider

$$
  \begin{cases}
  & \min\quad 2 \\
  & s.t \quad x_1=1 \qquad (P) \\
  & \qquad x_1,x_2 \geq 0,
  \end{cases}
  \qquad\qquad
  \begin{cases}
  & \max\quad w_1 \\
  & s.t \quad w_1 \leq 2 \qquad (D)
  \end{cases}
$$

The dual has an unique non-degenerated optimal solution.

**d)** It is possible.

Consider the primal-dual problem pair,

$$
  \begin{cases}
  & \min\quad 0 \\
  & s.t \quad -x_1 + x_2 \geq 0 \qquad (P) \\
  & \qquad x_2 \geq 1 \\
  & \qquad x_1,x_2 \geq 0,
  \end{cases}
  \qquad\qquad
  \begin{cases}
  & \max\quad w_2 \\
  & s.t \quad w_1 \leq 0 \qquad (D)\\
  & s.t \quad w_2 \leq 0 \\
   & \qquad w_1,w_2 \geq 0 \\
  \end{cases}
$$

$(x_1,x_2)=(0,0)$ is an infeasible basic solution. And the pair is $(0,0)-(0,0)$, where shares the same objective value 0.

**e)** **False.** If the primal is unbounded, its dual must be infeasible. But dual feasibility does not depend on $b$.

## 6.23
Denote the i-th row of the constrain matrix $A$ is $a_i$.

$$
\begin{cases}
& \min \quad c^Tx\\
& s.t \quad Ax=b \\
& \qquad x\geq 0
\end{cases}
\qquad
\begin{cases}
& \max \quad b^Tw\\
& s.t \quad A^Tw\leq c\\
\end{cases}
$$

**a)** Since this operation won't change the feasible set of the primal problem, hence the optimal value of the primal and dual respectively. The k-th element of the optimal solution of the dual problem will be divided by $\lambda$.

**b)** Assume that we add $\lambda a_j$ to $a_i$. Since this operation won't change the feasible set of the primal problem, hence the optimal value of the primal and dual respectively. The transformed constrain matrix $A_1=E_{ij}(\lambda)A$ and $b_1=E_{ij}(\lambda)b$, where $E_{ij}(\lambda)$ only differs from the identity matrix in the (i,j) element, whose element is $\lambda$ rather than $0$. To maintain the feasibility and optimality, the optimal solution $\tilde w$of the changed dual problem must satisfy

$${\tilde w}^T E_{ij}(\lambda) A\leq c^T \text { and }b_1^T  \tilde w=b^Tw.$$

Set ${\tilde w}^T E_{ij}(\lambda)=w^T$, we have ${\tilde w}^T = w^TE_{ij}(-\lambda)$. And it achieves the optimality.

To me more specific, $$\tilde w_k=w_k, k \neq i \text{ and } \tilde w_i = w_i - \lambda w_j.$$

**c)**

* If the modification does not change any column in the basis $B$. Then the optimal solution of the primal won't change.

* If the modification does change the  column in the basis $B$, then the optimal solution of the primal will change. Suppose, we add $\lambda A_{[,j]}$ to $\lambda A_{[,i]}$, then
$\tilde B = B E_{ji}(\lambda).$ Therefore we have

$$
x^{new} = E_{ji}(-\lambda)x^{old}.
$$

We can say nothing about the dual solutions.

## 6.27

**a)** The dual can be written as

$$
\begin{align}
& \max \quad l\\
& s.t \quad w_1 + w_2 = 1\\
& \qquad l - 2w_1 + 3w_2 \leq 0\\
& \qquad l + w_1 - 2w_2 \leq 0\\
& \qquad l - w_2 \leq 0\\
& \qquad w_1, w_2 \geq 0\\
& \qquad l \text{ is free}
\end{align}
$$

**b)** The iterpretation is as follow.

$w_1, w_2$ are the probabilities that the row player will select corresponding strategy. Then the problme is to maximum the minimal extepect  payoff to the row player.

**c)** This problem can be coverted to

$$
\max_{w_2 \in [0, 1]} \min \{2-5w_2, 3w_2 -1, w_2 \}.
$$

So the optimal solution is $(l, w_1,w_2)= (1/8, 5/8, 3/8)$.

**d)**
By the complementary slackness condition and strong duality, we have

$$(z,x_1,x_2,x_3)= (1/8, 3/8, 5/8, 0)$$

**e)** For both row and column player they put a larger weight (5/8) on the strategy that they will earn on expectation and smaller weight on the strategy that will lead to lose. Moever, the column palyer place a zero weight on the strategy will lose anyway.

## 6.35

**a)**

$\Rightarrow$: suppose $x_j=0$ for all optimal primal solutions.

Then the proof is same as the one given in the Exercise 4.20 **c)**. Please refer to it, which appears in the following.

$\Leftarrow$: suppose a complementary dual variable is positive for some optimal dual solutions.

Since the complementary slackness conditions hold for all primal and dual optimal solutions. Since for some $x_j$ is positive in the optimal dual, then its complementary primal variable has to be positive.

**b)**

If $x_j$ is unbounded, then Columns $A_j\leq 0$, then since $w\geq 0$, we have $w^T A_j\leq 0$. $w_{j+m}= c_j - w^T A_j$ is $x_j$'s complementary variable. By the dual feasibility, we know $w_{j+m}\geq 0$, and since $w^T A_j\leq 0$, we have $w_{j+m}\leq - w^T A_j$.

The argument cab be reversed if $w_{j+m}$ is bounded.

**c)**
Suppose $x_j$ is unbounded. Since it is optimal, by complementary slackness condition, we have $(c_j - w^T A_j)x_j=0$. As $x_j$ is unbounded, we have $w_{j+m}= c_j - w^T A_j=0$.

Suppose $w_{j+m}= c_j - w^T A_j=0$, then $x_j$ is unbounded.

**d)**

## Exercise 4.5

**a)** **True**.

Since there is no $c_B^TB^{-1}$ is feasible, we know that $x^*$ is not optimal yet. So the optimal cost is strictly smaller than $c^Tx^*$.

**b)** **True**.

The whole purpose of the auxiliary primal problem is to create a problem that is feasible. Since the optimal cost of the auxiliary primal problem is bounded by 0, so the dual is feasible.

**c)** **True**.

Eliminate the equality constraint will remove the corresponding dual variable, which is equivalent to set it as 0 in the constraint.

**d)** **True**.

This is obvious by weak duality theorem.

## Exercise 4.7

**a)** The primal problem can be written as

$$
\begin{align}
& \min \quad z\\
& s.t \quad 1_m^T z -Ax \geq -b\\
& \qquad z,x \text{ is free.}
\end{align}
$$

The corresponding dual is

$$
\begin{align}
& \max \quad (-b)^Tw\\
& s.t \quad 1_m^Tw = 1\\
& \qquad A^Tw = 0_m
\end{align}
$$

By the strong duality theorem, we have $(-b)^Tw^* = v,$ where $w^*$ is the optimal solution for the dual. Hence, for any $w$ satisfy the constraints given in the dual problem, one must have $$-w^Tb \leq v.$$

**b)** By strong duality theorem, we have the optimal cost of the dual is $v$.

## Exercise 4.15

* Dual Problem:

$$
\begin{align}
& \max \quad w\\
& s.t \quad w \leq 1
\end{align}
$$

* The optimal solutions for the primal are $\{(x_1,x_2)|x_1\geq 0, x_2 = 1\}$, which implies that it doesn't have an unique optimal solution. But it has non-degenerate solutions as long as $x_1 >0$.

* The optimal solution for the dual is $w=1$, which implies that it do have an unique optimal solution. Also, it has a non-degenerate solution.

---
1. The primal problem does not have a non-degenerate BFS.

2. The dual problem has a non-degenerate BFS, but the optimal solution for the primal is not unique.

From 1 and 2, we know this is not in agreement with the aforementioned statement.

## Exercise 4.20

**a)**
Denote

$$
\begin{cases}
& \min \quad c^Tx\\
& s.t \quad Ax=b  \qquad (P)\\
& \qquad x\geq 0
\end{cases}
\qquad
\begin{cases}
& \max \quad b^Tw\\
& s.t \quad A^Tw\leq c  \qquad (D)
\end{cases}
$$

$$
\begin{cases}
& \min \quad -x_j\\
& s.t \quad Ax=b  \qquad (P')\\
&\qquad c^Tx \leq d\\
& \qquad x\geq 0
\end{cases}
\qquad
\begin{cases}
& \max \quad b^Tw+dt\\
& s.t \quad A^Tw + tc^T \leq -e_j  \qquad (D')\\
& \qquad t\leq 0
\end{cases}
$$

Suppose $x^*$ is the optimal solution for (P), then it provides an upper bound on (P'). Note that we we have $x\geq 0$ and $x_j^*=0$, we know that the optimal value for (P') is also zero. Therefore, the optimal value for (D') is also 0.

Suppose $(w^*, t^*)$ is an optimal solution for (D') and $w_0$ is an optimal solution for (D).

* $t^*=0$, then we have $$b^Tw^* = 0 , A^Tw^* \leq -e_j.$$ Note that $$b^T(w^* + w_0)=b^Tw_0 \text{ and } A^T(w^*+w_0) \leq c - e_j.$$ We assert that $w^*+w_0$ is another optimal solution for (D). And specifically, we know that $$(w^* + w_0)^TA_j \leq c_j - 1 < c_j.$$

* $t^*<0$, then denote $\bar w = \frac{-w^*}{t^*}$. We have $b^tw^* + dt^*=0$ and $A^Tw^* + t^*c^T \leq - e_j$, which implies $$b^T\bar w = d \text{ and } A^T\bar w + \frac{e_j}{t^*}\leq c^T.$$ So $\bar w$ is an optimal solution for (D). Notice that $$\bar w^T A_j = c_j + \frac{1}{t^*} < c_j.$$

Two bullet points conclude the proof.

**b)**

From **a)**, we know that there exist $w^{(j)}, j\in J=\{j| x_j=0, j=1,2,...,n\}$ s.t $w^{(j)}$  is an optimal solution for (D) and ${w^{(j)}}^TA_j < c_j$. Denote $W^{**}= \frac{1}{|J|}\sum_{j\in J} w^{(j)}$, then we have   ${W^{**}}^TA_j < c_j, j\in J$.

Furthermore, we have $x_j >0, j \notin J.$

**c)**

i) Consider

$$
\begin{cases}
& \min \quad -x_j\\
& s.t \quad Ax\geq b  \qquad (P')\\
&\qquad -c^Tx \geq -d\\
& \qquad x\geq 0
\end{cases}
\qquad
\begin{cases}
& \max \quad b^Tw+dt\\
& s.t \quad w^TA - tc^T \leq -e_j^T  \qquad (D')\\
& \qquad w, t\geq 0
\end{cases}
$$

And consider for all $j=1,2,...,n$, $x_j=0$ and $x_j>0$ cases. Use the similar arguments in **a)** and **b)**, we concludes the proof.

ii) We can treat the dual problem as the primal and re-write is as

$$
  \begin{cases}
  & \max\quad b^T w \\
  & s.t \quad A^Tw\leq c \qquad (P) \\
  & \qquad w \geq 0,
  \end{cases}
  \qquad\qquad
  \begin{cases}
  & \min\quad x^Tc \\
  & s.t \quad x^T A^T\geq b^T \qquad (D) \\
  & \qquad x \geq 0,
  \end{cases}
$$

and use the similar method from i), we conclude the proof.

**d)**

By KKT condition, we know that $(w_1^*,w_2^*)=(5,0)$. And ${w^*}^TA < c^T$

The optimal solutions lie on the segment $\{(x_1,x_2)| 2/3 \leq x_1 \leq 2, x_2 = 2 - x_1\}$ for the primal.
As for the optimal solution for the dual, it is always $(5,0)$.

Pairs that satisfy strictly complementary slackness conditions:

$$(2/3, 4/3) \sim (5, 0) \qquad (2,0) \sim (5,0)$$

## Exercise 4.27
By the strictly complementary slackness condition, **a)** and **b)** are equivalent.