## Definitions

### Observables
+ time $t_{k}$
+ position $m_{k}$
+ velocity $v_{k}$
+ LFP power $y_{k}$
+ population spiking activity of $C$ cells $\Delta N_{k}^{1:C}$

### (Semi-)Latent Variables
+ replay indicator
$
  \begin{equation}
    I_{k} =
    \begin{cases}
      0, & \text{if no replay at}\ t_{k} \\
      1, & \text{if replay at}\ t_{k} \\
    \end{cases}
  \end{equation}
$
+ position/replay position
$
  \begin{equation}
    x_{k} =
    \begin{cases}
      \text{unobserved replay position}, & \text{if}\ I_{k} = 1 \\
      m_{k}, & \text{if}\ I_{k} = 0 \\
    \end{cases}
  \end{equation}
$

## Filter

### State Transitions
+ replay state transition $\Pr(I_{k} \mid I_{k-1}, v_{k-1})$
+ movement state transition
$
  \begin{equation}
    p(x_{k} \mid x_{k-1}, I_{k}, I_{k-1}) =
    \begin{cases}
      \delta (m_{k}), & \text{if}\ I_{k} = 0 \\
      U(a, b), & \text{if}\ I_{k} = 1 \text{ and }\ I_{k-1} = 0 \\
      N(x_{k-1}, \sigma), & \text{if}\ I_{k} = 1 \text{ and }\ I_{k-1} = 1 \\
    \end{cases}
  \end{equation}
$

### Likelihoods
+ velocity likelihood $p(v_{k} \mid I_{k}, v_{k})$
+ LFP power likelihood $p(y_{k} \mid I_{k}, y_{k})$
+ population spiking activity likelihood $p(\Delta N_{k}^{1:C} \mid I_{k}, )$

### Joint Filter Distribution
+ $p(I_{k}, x_{k} \mid y_{1:k}, \Delta N_{1:k}^{1:C}, v_{1:k})$

### Calculation Goals
+ replay probability $\Pr(I_{k} = 1 \mid y_{1:k}, \Delta N_{1:k}^{1:C}, v_{1:k})$
+ replay position $p(x_{k}, I_{k} = 1\mid y_{1:k}, \Delta N_{1:k}^{1:C}, v_{1:k})$

## Filter Derivation

1. Bayes ($ posterior \propto likelihood * prior $)
$$
p(I_{k}, x_{k} \mid y_{1:k}, \Delta N_{1:k}^{1:C}, v_{1:k}) \propto
p(y_{k} \mid I_{k}, x_{k}, y_{1:k-1}, \Delta N_{1:k}^{1:C}, v_{1:k}) *
p(\Delta N_{k}^{1:C} \mid I_{k}, x_{k}, y_{1:k}, \Delta N_{1:k-1}^{1:C}, v_{1:k}) *
p(v_{k} \mid I_{k}, x_{k}, y_{1:k-1}, \Delta N_{1:k-1}^{1:C}, v_{1:k-1}) *
p(I_{k}, x_{k} \mid y_{1:k-1}, \Delta N_{1:k-1}^{1:C}, v_{1:k-1})
$$

2. Simplify likelihoods
$$
p(I_{k}, x_{k} \mid y_{1:k}, \Delta N_{1:k}^{1:C}, v_{1:k})  \propto
p(y_{k} \mid I_{k}, y_{1:k-1}) *
p(\Delta N_{k}^{1:C} \mid x_{k}, \Delta N_{1:k-1}^{1:C}) *
p(v_{k} \mid I_{k}, v_{1:k-1}) *
p(I_{k}, x_{k} \mid y_{1:k-1}, \Delta N_{1:k-1}^{1:C}, v_{1:k-1})
$$

3. Chapman-Kolmogorov to define prior in terms of previous time step's posterior
$$
\begin{align*}
p(I_{k}, x_{k} \mid y_{1:k-1}, \Delta N_{1:k-1}^{1:C}, v_{1:k-1}) &= \sum_{I_{k-1}} \int p(I_{k}, I_{k-1}, x_{k}, x_{k-1} \mid y_{1:k-1}, \Delta N_{1:k-1}^{1:C}, v_{1:k-1}) * dx_{k-1}
\\ &= \sum_{I_{k-1}} \int p(I_{k}, x_{k} \mid I_{k-1}, x_{k-1}, y_{1:k-1}, \Delta N_{1:k-1}^{1:C}, v_{1:k-1}) * p(I_{k-1}, x_{k-1} \mid y_{1:k-1}, \Delta N_{1:k-1}^{1:C}, v_{1:k-1}) * dx_{k-1} \tag{Markov}
\\ &= \sum_{I_{k-1}} \int p(I_{k}, x_{k} \mid I_{k-1}, x_{k-1}) * p(I_{k-1}, x_{k-1} \mid y_{1:k-1}, \Delta N_{1:k-1}^{1:C}, v_{1:k-1}) * dx_{k-1} \tag{Markov}
\\ &= \sum_{I_{k-1}} \int p(x_{k} \mid x_{k-1}, I_{k}, I_{k-1}) * \Pr(I_{k} \mid I_{k-1}, v_{k-1}) *
p(I_{k-1}, x_{k-1} \mid y_{1:k-1}, \Delta N_{1:k-1}^{1:C}, v_{1:k-1}) * dx_{k-1}
\end{align*}$$

4. Final Filter
$$
p(I_{k}, x_{k} \mid y_{1:k}, \Delta N_{1:k}^{1:C}, v_{1:k})  \propto
p(y_{k} \mid I_{k}, y_{1:k-1}) *
p(\Delta N_{k}^{1:C} \mid x_{k}, \Delta N_{1:k-1}^{1:C}) *
p(v_{k} \mid I_{k}, v_{1:k-1}) *
\sum_{I_{k-1}} \int p(x_{k} \mid x_{k-1}, I_{k}, I_{k-1}) * \Pr(I_{k} \mid I_{k-1}, v_{k-1}) *
p(I_{k-1}, x_{k-1} \mid y_{1:k-1}, \Delta N_{1:k-1}^{1:C}, v_{1:k-1}) * dx_{k-1}
$$

## Computing goals from filter

+ replay position
$$
\begin{align*}
p(x_{k} \mid I_{k} = 1, y_{1:k}, \Delta N_{1:k}^{1:C}, v_{1:k}) &= \frac{p(x_{k}, I_{k} = 1 \mid y_{1:k}, \Delta N_{1:k}^{1:C}, v_{1:k})}{\Pr(I_{k} = 1 \mid y_{1:k}, \Delta N_{1:k}^{1:C}, v_{1:k})}  \tag{Def. of conditional probability}
\\
\\ p(x_{k}, I_{k} = 1 \mid y_{1:k}, \Delta N_{1:k}^{1:C}, v_{1:k}) &\propto p(y_{k} \mid I_{k} = 1, y_{1:k-1}) *
p(\Delta N_{k}^{1:C} \mid x_{k}, \Delta N_{1:k-1}^{1:C}) *
p(v_{k} \mid I_{k} = 1, v_{1:k-1}) *
\sum_{I_{k-1}} \int p(x_{k} \mid x_{k-1}, I_{k} = 1, I_{k-1}) * \Pr(I_{k} = 1 \mid I_{k-1}, v_{k-1}) *
p(I_{k-1}, x_{k-1} \mid y_{1:k-1}, \Delta N_{1:k-1}^{1:C}, v_{1:k-1}) * dx_{k-1}
\\ &\propto \text{likelihoods} * \Bigg(
\int p(x_{k} \mid x_{k-1}, I_{k} = 1, I_{k-1} = 0) * \Pr(I_{k} = 1 \mid I_{k-1} = 0, v_{k-1}) *
p(I_{k-1} = 0, x_{k-1} \mid y_{1:k-1}, \Delta N_{1:k-1}^{1:C}, v_{1:k-1}) * dx_{k-1} +
\int p(x_{k} \mid x_{k-1}, I_{k} = 1, I_{k-1} = 1) * \Pr(I_{k} = 1 \mid I_{k-1} = 1, v_{k-1}) *
p(I_{k-1} = 1, x_{k-1} \mid y_{1:k-1}, \Delta N_{1:k-1}^{1:C}, v_{1:k-1}) * dx_{k-1} \Bigg)
\\ &\propto \text{likelihoods} * \Bigg( \Pr(I_{k} = 1 \mid I_{k-1} = 0, v_{k-1}) * \int U(a, b) *
p(I_{k-1} = 0, x_{k-1} \mid y_{1:k-1}, \Delta N_{1:k-1}^{1:C}, v_{1:k-1}) * dx_{k-1} +
\Pr(I_{k} = 1 \mid I_{k-1} = 1, v_{k-1}) * \int N(x_{k-1}, \sigma) *
p(I_{k-1} = 1, x_{k-1} \mid y_{1:k-1}, \Delta N_{1:k-1}^{1:C}, v_{1:k-1}) * dx_{k-1} \Bigg)
\end{align*}$$

### Computation

#### Initialization

+ $Pr(I_{0} = 0) = 1$ Assume we are not in a replay at the first timestep
+ $p(x_{0}, I_{0} = 0) = \delta(m_{0}) $

#### Predict Step
Predict prior = $p(x_{k}, I_{k} \mid H_{k-1}) = \sum_{I_{k-1}} \int p(x_{k} \mid x_{k-1}, I_{k}, I_{k-1}) * \Pr(I_{k} \mid I_{k-1}, v_{k-1}) * p(x_{k-1}, I_{k-1}) dx_{k-1}$

+ $I_{k} = 0, I_{k-1} = 0$
$$
\begin{align*}
p(x_{k}, I_{k} = 0, I_{k-1} = 0 \mid H_{1:k-1}) &= \int p(x_{k} \mid x_{k-1}, I_{k-1} = 0, I_{k} = 0) * Pr(I_{k} = 0 \mid I_{k-1} = 0, v_{k-1}) * p(x_{k-1}, I_{k-1} = 0 \mid H_{1:k-1}) dx_{k-1}
\\ &= \int \delta(m_{k}) * Pr(I_{k} = 0 \mid I_{k-1} = 0, v_{k-1}) * p(x_{k-1}, I_{k-1} = 0 \mid H_{1:k-1}) dx_{k-1}
\\ &= \delta(m_{k}) * Pr(I_{k} = 0 \mid I_{k-1} = 0, v_{k-1}) * \int  p(x_{k-1}, I_{k-1} = 0 \mid H_{1:k-1}) dx_{k-1}
\\ &= \delta(m_{k}) * Pr(I_{k} = 0 \mid I_{k-1} = 0, v_{k-1}) * Pr(I_{k-1} = 0)
\\ &= \delta(m_{k}) * \big(1 - Pr(I_{k} = 1 \mid I_{k-1} = 0, v_{k-1})\big) * Pr(I_{k-1} = 0)
\end{align*}
$$

+ $I_{k} = 0, I_{k-1} = 1$
$$
\begin{align*}
p(x_{k}, I_{k} = 0, I_{k-1} = 1 \mid H_{1:k-1}) &= \int p(x_{k} \mid x_{k-1}, I_{k-1} = 1, I_{k} = 0) * Pr(I_{k} = 0 \mid I_{k-1} = 1, v_{k-1}) * p(x_{k-1}, I_{k-1} = 1 \mid H_{1:k-1}) dx_{k-1}
\\ &= \int \delta(m_{k}) * Pr(I_{k} = 0 \mid I_{k-1} = 1, v_{k-1}) * p(x_{k-1}, I_{k-1} = 1 \mid H_{1:k-1}) dx_{k-1}
\\ &= \delta(m_{k}) * Pr(I_{k} = 0 \mid I_{k-1} = 1, v_{k-1}) * \int  p(x_{k-1}, I_{k-1} = 1 \mid H_{1:k-1}) dx_{k-1}
\\ &= \delta(m_{k}) * Pr(I_{k} = 0 \mid I_{k-1} = 1, v_{k-1}) * Pr(I_{k-1} = 1)
\\ &= \delta(m_{k}) * \big(1 - Pr(I_{k} = 1 \mid I_{k-1} = 1, v_{k-1})\big) * Pr(I_{k-1} = 1)
\end{align*}
$$

+ $I_{k} = 1, I_{k-1} = 0$
$$
\begin{align*}
p(x_{k}, I_{k} = 1, I_{k-1} = 0 \mid H_{1:k-1}) &= \int p(x_{k} \mid x_{k-1}, I_{k-1} = 0, I_{k} = 1) * Pr(I_{k} = 1 \mid I_{k-1} = 0, v_{k-1}) * p(x_{k-1}, I_{k-1} = 0 \mid H_{1:k-1}) dx_{k-1}
\\ &= \int \mathcal{U}(a, b) * Pr(I_{k} = 1 \mid I_{k-1} = 0, v_{k-1}) * p(x_{k-1}, I_{k-1} = 0 \mid H_{1:k-1}) dx_{k-1}
\\ &= \mathcal{U}(a, b) * Pr(I_{k} = 1 \mid I_{k-1} = 0, v_{k-1}) * \int p(x_{k-1}, I_{k-1} = 0 \mid H_{1:k-1}) dx_{k-1}
\\ &= \mathcal{U}(a, b) * Pr(I_{k} = 1 \mid I_{k-1} = 0, v_{k-1}) * Pr(I_{k-1} = 0)
\end{align*}
$$

+ $I_{k} = 1, I_{k-1} = 1$
$$
\begin{align*}
p(x_{k}, I_{k} = 1, I_{k-1} = 1 \mid H_{1:k-1}) &= \int p(x_{k} \mid x_{k-1}, I_{k-1} = 1, I_{k} = 1) * Pr(I_{k} = 1 \mid I_{k-1} = 1, v_{k-1}) * p(x_{k-1}, I_{k-1} = 1 \mid H_{1:k-1}) dx_{k-1}
\\ &= \int \mathcal{N}(x_{k-1}, \hat{\sigma}) * Pr(I_{k} = 1 \mid I_{k-1} = 1, v_{k-1}) * p(x_{k-1}, I_{k-1} = 1 \mid H_{1:k-1}) dx_{k-1}
\\ &= Pr(I_{k} = 1 \mid I_{k-1} = 1, v_{k-1}) * \int \mathcal{N}(x_{k-1}, \hat{\sigma}) * p(x_{k-1}, I_{k-1} = 1 \mid H_{1:k-1}) dx_{k-1}
\end{align*}
$$

So we have:

$$
p(x_{k}, I_{k} = 0 \mid H_{1:k-1}) = \delta(m_{k}) * \big[\big(1 - Pr(I_{k} = 1 \mid I_{k-1} = 0, v_{k-1})\big) * Pr(I_{k-1} = 0) + \big(1 - Pr(I_{k} = 1 \mid I_{k-1} = 1, v_{k-1})\big) * Pr(I_{k-1} = 1)\big]
$$

and

$$
p(x_{k}, I_{k} = 1 \mid H_{1:k-1}) = \mathcal{U}(a, b) * Pr(I_{k} = 1 \mid I_{k-1} = 0, v_{k-1}) * Pr(I_{k-1} = 0) + Pr(I_{k} = 1 \mid I_{k-1} = 1, v_{k-1}) * \int \mathcal{N}(x_{k-1}, \hat{\sigma}) * p(x_{k-1}, I_{k-1} = 1 \mid H_{1:k-1}) dx_{k-1}
$$

### Update Step

$$
p(x_{k}, I_{k} \mid H_{1:k}) = Normalize\Big(p(x_{k}, I_{k} \mid H_{k-1}) * likelihoods_{k}\Big)
$$

### Algorithm

Set initial conditions:
+ $Pr(I_{0} = 0) = 1$
+ $p(x_{0}, I_{0} = 0) = \delta(m_{0}) $

For each time step $k$:
1. Predict prior
    $$
    p(x_{k}, I_{k} = 0 \mid H_{1:k-1}) = \delta(m_{k}) * \big[\big(1 - Pr(I_{k} = 1 \mid I_{k-1} = 0, v_{k-1})\big) * Pr(I_{k-1} = 0) + \big(1 - Pr(I_{k} = 1 \mid I_{k-1} = 1, v_{k-1})\big) * Pr(I_{k-1} = 1)\big]
    $$

    and

    $$
    p(x_{k}, I_{k} = 1 \mid H_{1:k-1}) = \mathcal{U}(a, b) * Pr(I_{k} = 1 \mid I_{k-1} = 0, v_{k-1}) * Pr(I_{k-1} = 0) + Pr(I_{k} = 1 \mid I_{k-1} = 1, v_{k-1}) * \int \mathcal{N}(x_{k-1}, \hat{\sigma}) * p(x_{k-1}, I_{k-1} = 1 \mid H_{1:k-1}) dx_{k-1}
    $$

2. Update Posterior

$$
p(x_{k}, I_{k} \mid H_{1:k}) = Normalize\Big(p(x_{k}, I_{k} \mid H_{k-1}) * likelihoods_{k}\Big)
$$

## Smoother

### Smoother State Transitions

+ Movement State Transition $p(x_{k+1} \mid x_{k}, I_{k}, I_{k+1})$

For the filter we fit:
$$
 \begin{equation}
    p(x_{k} \mid x_{k-1}, I_{k}, I_{k-1}) =
    \begin{cases}
      \delta(m_{k}), & \text{if}\ I_{k} = 0, I_{k-1} \in \{0, 1\} \\
      \mathcal{U}(a, b), & \text{if}\ I_{k} = 1, I_{k-1} = 0 \\
      \mathcal{N}(x_{k-1}, \hat{\sigma}), & \text{if}\ I_{k} = 1, I_{k-1} = 1 \\
    \end{cases}
  \end{equation}
$$

So

$$
 \begin{equation}
    p(x_{k + 1} \mid x_{k}, I_{k+1}, I_{k}) =
    \begin{cases}
      \delta(m_{k+1}), & \text{if}\ I_{k+1} = 0, I_{k} \in \{0, 1\} \\
      \mathcal{U}(a, b), & \text{if}\ I_{k+1} = 1, I_{k} = 0 \\
      \mathcal{N}(x_{k}, \hat{\sigma}), & \text{if}\ I_{k+1} = 1, I_{k} = 1 \\
    \end{cases}
  \end{equation}
$$

+ Replay State Transition $Pr(I_{k + 1} \mid I_{k}, v_{k})$

For the filter, we fit $Pr(I_{k} \mid I_{k-1}, v_{k-1})$, so we can just use it evaluated at $k+1$

### Predict Step
Predict the prior distribution $p(x_{k + 1}, I_{k + 1} \mid H_{1:k})$ using the filter distribution $p(x_{k}, I_{k} \mid H_{1:k})$ and state transition $p(x_{k+1} \mid x_{k}, I_{k}, I_{k+1}) * Pr(I_{k + 1} \mid I_{k}, v_{k}) $:
$$prior = p(x_{k + 1}, I_{k + 1} \mid H_{1:k}) = \sum_{I_{k}} \int p(x_{k+1} \mid x_{k}, I_{k}, I_{k+1}) * Pr(I_{k + 1} \mid I_{k}, v_{k}) * p(x_{k}, I_{k} \mid H_{1:k}) dx_{k}
$$

+ $I_{k + 1} = 0, I_{k} = 0$
$$
\begin{align*}
p(x_{k + 1}, I_{k + 1} = 0, I_{k} = 0 \mid H_{1:k}) &= \int p(x_{k+1} \mid x_{k}, I_{k} = 0, I_{k+1} = 0) * Pr(I_{k + 1} = 0 \mid I_{k} = 0, v_{k}) * p(x_{k}, I_{k} = 0 \mid H_{1:k}) dx_{k}
\\ &= \int \delta(m_{k+1}) * Pr(I_{k + 1} = 0 \mid I_{k} = 0, v_{k}) * p(x_{k}, I_{k} = 0 \mid H_{1:k}) dx_{k}
\\ &= \delta(m_{k+1}) * Pr(I_{k + 1} = 0 \mid I_{k} = 0, v_{k}) * \int  p(x_{k}, I_{k} = 0 \mid H_{1:k}) dx_{k}
\\ &= \delta(m_{k+1}) * Pr(I_{k + 1} = 0 \mid I_{k} = 0, v_{k}) * Pr(I_{k} = 0)
\\ &= \delta(m_{k+1}) * \big(1 - Pr(I_{k + 1} = 1 \mid I_{k} = 0, v_{k})\big) * Pr(I_{k} = 0)
\end{align*}
$$

+ $I_{k + 1} = 0, I_{k} = 1$
$$
\begin{align*}
p(x_{k + 1}, I_{k + 1} = 0, I_{k} = 1 \mid H_{1:k}) &= \int p(x_{k+1} \mid x_{k}, I_{k} = 1, I_{k+1} = 0) * Pr(I_{k + 1} = 0 \mid I_{k} = 1, v_{k}) * p(x_{k}, I_{k} = 1 \mid H_{1:k}) dx_{k}
\\ &= \int \delta(m_{k+1}) * Pr(I_{k + 1} = 0 \mid I_{k} = 1, v_{k}) * p(x_{k}, I_{k} = 1 \mid H_{1:k}) dx_{k}
\\ &= \delta(m_{k+1}) * Pr(I_{k + 1} = 0 \mid I_{k} = 1, v_{k}) * \int  p(x_{k}, I_{k} = 1 \mid H_{1:k}) dx_{k}
\\ &= \delta(m_{k+1}) * Pr(I_{k + 1} = 0 \mid I_{k} = 1, v_{k}) * Pr(I_{k} = 1)
\\ &= \delta(m_{k+1}) * \big(1 - Pr(I_{k + 1} = 1 \mid I_{k} = 1, v_{k})\big) * Pr(I_{k} = 1)
\end{align*}
$$

+ $I_{k + 1} = 1, I_{k} = 0$
$$
\begin{align*}
p(x_{k + 1}, I_{k + 1} = 1, I_{k} = 0 \mid H_{1:k}) &= \int p(x_{k+1} \mid x_{k}, I_{k} = 0, I_{k+1} = 1) * Pr(I_{k + 1} = 1 \mid I_{k} = 0, v_{k}) * p(x_{k}, I_{k} = 0 \mid H_{1:k}) dx_{k}
\\ &= \int \mathcal{U}(a, b) * Pr(I_{k + 1} = 1 \mid I_{k} = 0, v_{k}) * p(x_{k}, I_{k} = 0 \mid H_{1:k}) dx_{k}
\\ &= \mathcal{U}(a, b) * Pr(I_{k + 1} = 1 \mid I_{k} = 0, v_{k}) * \int p(x_{k}, I_{k} = 0 \mid H_{1:k}) dx_{k}
\\ &= \mathcal{U}(a, b) * Pr(I_{k + 1} = 1 \mid I_{k} = 0, v_{k}) * Pr(I_{k} = 0)
\end{align*}
$$

+ $I_{k + 1} = 1, I_{k} = 1$
$$
\begin{align*}
p(x_{k + 1}, I_{k + 1} = 1, I_{k} = 1 \mid H_{1:k}) &= \int p(x_{k+1} \mid x_{k}, I_{k} = 1, I_{k+1} = 1) * Pr(I_{k + 1} = 1 \mid I_{k} = 1, v_{k}) * p(x_{k}, I_{k} = 1 \mid H_{1:k}) dx_{k}
\\ &= \int \mathcal{N}(x_{k}, \hat{\sigma}) * Pr(I_{k + 1} = 1 \mid I_{k} = 1, v_{k}) * p(x_{k}, I_{k} = 1 \mid H_{1:k}) dx_{k}
\\ &= Pr(I_{k + 1} = 1 \mid I_{k} = 1, v_{k}) * \int \mathcal{N}(x_{k}, \hat{\sigma}) * p(x_{k}, I_{k} = 1 \mid H_{1:k}) dx_{k}
\end{align*}
$$

So we have:

$$
p(x_{k + 1}, I_{k + 1} = 0 \mid H_{1:k}) = \delta(m_{k+1}) * \big[\big(1 - Pr(I_{k + 1} = 1 \mid I_{k} = 0, v_{k})\big) * Pr(I_{k} = 0) + \big(1 - Pr(I_{k + 1} = 1 \mid I_{k} = 1, v_{k})\big) * Pr(I_{k} = 1)\big]
$$

and

$$
p(x_{k + 1}, I_{k + 1} = 1 \mid H_{1:k}) = \mathcal{U}(a, b) * Pr(I_{k + 1} = 1 \mid I_{k} = 0, v_{k}) * Pr(I_{k} = 0) + Pr(I_{k + 1} = 1 \mid I_{k} = 1, v_{k}) * \int \mathcal{N}(x_{k}, \hat{\sigma}) * p(x_{k}, I_{k} = 1 \mid H_{1:k}) dx_{k}
$$

## Backwards Update Step

Update the smoother distribution $p(x_{k}, I_{k} \mid H_{1:T})$ using the filter distribution $p(x_{k}, I_{k} \mid H_{1:k})$, prior $p(x_{k + 1}, I_{k + 1} \mid H_{1:k})$ and state transition $p(x_{k+1} \mid x_{k}, I_{k}, I_{k+1}) * Pr(I_{k + 1} \mid I_{k}, v_{k}) $:
$$
p(x_{k}, I_{k} \mid H_{1:T}) = p(x_{k}, I_{k} \mid H_{1:k}) * weights
$$

where:
$$
weights = \sum_{I_{k+1}} \int \Big[ \frac{p(x_{k+1} \mid x_{k}, I_{k}, I_{k+1}) * Pr(I_{k + 1} \mid I_{k}, v_{k}) * p(x_{k+1}, I_{k+1} \mid H_{1:T})}{p(x_{k + 1}, I_{k + 1} \mid H_{1:k})} \Big] dx_{k+1}
$$

+ $I_{k + 1} = 0, I_{k} = 0$
$$
\begin{align*}
weights &= \int \Big[ \frac{p(x_{k+1} \mid x_{k}, I_{k} = 0, I_{k+1} = 0) * Pr(I_{k + 1} = 0 \mid I_{k} = 0, v_{k}) * p(x_{k+1}, I_{k+1} = 0 \mid H_{1:T})}{p(x_{k + 1}, I_{k + 1} = 0 \mid H_{1:k})} \Big] dx_{k+1}
\\ &= \int \Big[ \frac{\delta(m_{k + 1}) * Pr(I_{k + 1} = 0 \mid I_{k} = 0, v_{k}) * p(x_{k+1}, I_{k+1} = 0 \mid H_{1:T})}{p(x_{k + 1}, I_{k + 1} = 0 \mid H_{1:k})} \Big] dx_{k+1}
\\ &= Pr(I_{k + 1} = 0 \mid I_{k} = 0, v_{k}) * \int \Big[ \frac{\delta(m_{k + 1}) * p(x_{k+1}, I_{k+1} = 0 \mid H_{1:T})}{p(x_{k + 1}, I_{k + 1} = 0 \mid H_{1:k})} \Big] dx_{k+1}
\\ &= \big(1 - Pr(I_{k + 1} = 1 \mid I_{k} = 0, v_{k}\big) * \int \Big[ \frac{\delta(m_{k + 1}) * p(x_{k+1}, I_{k+1} = 0 \mid H_{1:T})}{p(x_{k + 1}, I_{k + 1} = 0 \mid H_{1:k})} \Big] dx_{k+1}
\\ &= \big(1 - Pr(I_{k + 1} = 1 \mid I_{k} = 0, v_{k}\big) * \frac{p(x_{k+1}=m_{k + 1}, I_{k+1} = 0 \mid H_{1:T})}{p(x_{k + 1}=m_{k + 1}, I_{k + 1} = 0 \mid H_{1:k})}
\end{align*}
$$

+ $I_{k + 1} = 0, I_{k} = 1$
$$
\begin{align*}
weights &= \int \Big[ \frac{p(x_{k+1} \mid x_{k}, I_{k} = 1, I_{k+1} = 0) * Pr(I_{k + 1} = 0 \mid I_{k} = 1, v_{k}) * p(x_{k+1}, I_{k+1} = 0 \mid H_{1:T})}{p(x_{k + 1}, I_{k + 1} = 0 \mid H_{1:k})} \Big] dx_{k+1}
\\ &= \int \Big[ \frac{\delta(m_{k + 1}) * Pr(I_{k + 1} = 0 \mid I_{k} = 1, v_{k}) * p(x_{k+1}, I_{k+1} = 0 \mid H_{1:T})}{p(x_{k + 1}, I_{k + 1} = 0 \mid H_{1:k})} \Big] dx_{k+1}
\\ &= Pr(I_{k + 1} = 0 \mid I_{k} = 1, v_{k}) * \int \Big[ \frac{\delta(m_{k + 1}) * p(x_{k+1}, I_{k+1} = 0 \mid H_{1:T})}{p(x_{k + 1}, I_{k + 1} = 0 \mid H_{1:k})} \Big] dx_{k+1}
\\ &= \big(1 - Pr(I_{k + 1} = 1 \mid I_{k} = 1, v_{k}\big) * \int \Big[ \frac{\delta(m_{k + 1}) * p(x_{k+1}, I_{k+1} = 0 \mid H_{1:T})}{p(x_{k + 1}, I_{k + 1} = 0 \mid H_{1:k})} \Big] dx_{k+1}
\\ &= \big(1 - Pr(I_{k + 1} = 1 \mid I_{k} = 1, v_{k}\big) * \frac{p(x_{k+1}=m_{k + 1}, I_{k+1} = 0 \mid H_{1:T})}{p(x_{k + 1}=m_{k + 1}, I_{k + 1} = 0 \mid H_{1:k})}
\end{align*}
$$

+ $I_{k + 1} = 1, I_{k} = 0$
$$
\begin{align*}
weights &= \int \Big[ \frac{p(x_{k+1} \mid x_{k}, I_{k} = 0, I_{k+1} = 1) * Pr(I_{k + 1} = 1 \mid I_{k} = 0, v_{k}) * p(x_{k+1}, I_{k+1} = 1 \mid H_{1:T})}{p(x_{k + 1}, I_{k + 1} = 1 \mid H_{1:k})} \Big] dx_{k+1}
\\ &= \int \Big[ \frac{\mathcal{U}(a, b) * Pr(I_{k + 1} = 1 \mid I_{k} = 0, v_{k}) * p(x_{k+1}, I_{k+1} = 1 \mid H_{1:T})}{p(x_{k + 1}, I_{k + 1} = 1 \mid H_{1:k})} \Big] dx_{k+1}
\\ &= \mathcal{U}(a, b) * Pr(I_{k + 1} = 1 \mid I_{k} = 0, v_{k}) * \int \Big[ \frac{p(x_{k+1}, I_{k+1} = 1 \mid H_{1:T})}{p(x_{k + 1}, I_{k + 1} = 1 \mid H_{1:k})} \Big] dx_{k+1}
\end{align*}
$$

+ $I_{k + 1} = 1, I_{k} = 1$
$$
\begin{align*}
weights &= \int \Big[ \frac{p(x_{k+1} \mid x_{k}, I_{k} = 1, I_{k+1} = 1) * Pr(I_{k + 1} = 1 \mid I_{k} = 1, v_{k}) * p(x_{k+1}, I_{k+1} = 1 \mid H_{1:T})}{p(x_{k + 1}, I_{k + 1} = 1 \mid H_{1:k})} \Big] dx_{k+1}
\\ &= \int \Big[ \frac{\mathcal{N}(x_{k}, \sigma) * Pr(I_{k + 1} = 1 \mid I_{k} = 1, v_{k}) * p(x_{k+1}, I_{k+1} = 1 \mid H_{1:T})}{p(x_{k + 1}, I_{k + 1} = 1 \mid H_{1:k})} \Big] dx_{k+1}
\\ &= Pr(I_{k + 1} = 1 \mid I_{k} = 1, v_{k}) * \int \Big[ \frac{\mathcal{N}(x_{k}, \sigma) * p(x_{k+1}, I_{k+1} = 1 \mid H_{1:T})}{p(x_{k + 1}, I_{k + 1} = 1 \mid H_{1:k})} \Big] dx_{k+1}
\end{align*}
$$

So we have:
+ $I_{k} = 0$
$$
\big(1 - Pr(I_{k + 1} = 1 \mid I_{k} = 0, v_{k}\big) * \frac{p(x_{k+1}=m_{k + 1}, I_{k+1} = 0 \mid H_{1:T})}{p(x_{k + 1}=m_{k + 1}, I_{k + 1} = 0 \mid H_{1:k})}
\\ + \mathcal{U}(a, b) * Pr(I_{k + 1} = 1 \mid I_{k} = 0, v_{k}) * \int \Big[ \frac{p(x_{k+1}, I_{k+1} = 1 \mid H_{1:T})}{p(x_{k + 1}, I_{k + 1} = 1 \mid H_{1:k})} \Big] dx_{k+1}
$$

+ $I_{k} = 1$
$$
\big(1 - Pr(I_{k + 1} = 1 \mid I_{k} = 1, v_{k}\big) * \frac{p(x_{k+1}=m_{k + 1}, I_{k+1} = 0 \mid H_{1:T})}{p(x_{k + 1}=m_{k + 1}, I_{k + 1} = 0 \mid H_{1:k})}
\\ + Pr(I_{k + 1} = 1 \mid I_{k} = 1, v_{k}) * \int \Big[ \frac{\mathcal{N}(x_{k}, \hat{\sigma}) * p(x_{k+1}, I_{k+1} = 1 \mid H_{1:T})}{p(x_{k + 1}, I_{k + 1} = 1 \mid H_{1:k})} \Big] dx_{k+1}
$$

### Algorithm

Set smoother posterior at time T to filter posterior at time T

$p(x_{T}, I_{T} \mid H_{1:T}) = p(x_{T}, I_{T} \mid H_{1:T})$

For each time step $k$:
1. Calculate Prior:
$
p(x_{k + 1}, I_{k + 1} = 0 \mid H_{1:k}) = \delta(m_{k+1}) * \big[\big(1 - Pr(I_{k + 1} = 1 \mid I_{k} = 0, v_{k})\big) * Pr(I_{k} = 0) + \big(1 - Pr(I_{k + 1} = 1 \mid I_{k} = 1, v_{k})\big) * Pr(I_{k} = 1)\big]
$
$
p(x_{k + 1}, I_{k + 1} = 1 \mid H_{1:k}) = \mathcal{U}(a, b) * Pr(I_{k + 1} = 1 \mid I_{k} = 0, v_{k}) * Pr(I_{k} = 0) + Pr(I_{k + 1} = 1 \mid I_{k} = 1, v_{k}) * \int \mathcal{N}(x_{k}, \hat{\sigma}) * p(x_{k}, I_{k} = 1 \mid H_{1:k}) dx_{k}
$
2. Compute Weights
    + $I_{k} = 0$
    $$
    \big(1 - Pr(I_{k + 1} = 1 \mid I_{k} = 0, v_{k}\big) * \frac{p(x_{k+1}=m_{k + 1}, I_{k+1} = 0 \mid H_{1:T})}{p(x_{k + 1}=m_{k + 1}, I_{k + 1} = 0 \mid H_{1:k})}
    \\ + \mathcal{U}(a, b) * Pr(I_{k + 1} = 1 \mid I_{k} = 0, v_{k}) * \int \Big[ \frac{p(x_{k+1}, I_{k+1} = 1 \mid H_{1:T})}{p(x_{k + 1}, I_{k + 1} = 1 \mid H_{1:k})} \Big] dx_{k+1}
    $$

    + $I_{k} = 1$
    $$
    \big(1 - Pr(I_{k + 1} = 1 \mid I_{k} = 1, v_{k}\big) * \frac{p(x_{k+1}=m_{k + 1}, I_{k+1} = 0 \mid H_{1:T})}{p(x_{k + 1}=m_{k + 1}, I_{k + 1} = 0 \mid H_{1:k})}
    \\ + Pr(I_{k + 1} = 1 \mid I_{k} = 1, v_{k}) * \int \Big[ \frac{\mathcal{N}(x_{k}, \hat{\sigma}) * p(x_{k+1}, I_{k+1} = 1 \mid H_{1:T})}{p(x_{k + 1}, I_{k + 1} = 1 \mid H_{1:k})} \Big] dx_{k+1}
    $$
3. Backwards Update Posterior and Normalize
$$
p(x_{k}, I_{k} \mid H_{1:T}) = Normalize\Big(p(x_{k}, I_{k} \mid H_{1:k}) * weights\Big)
$$

```python

```


## Authors: Benjamin Barinka, Snigdhayan Mahanta, Matthias Semmelmann (in alphabetical order of last name)

In this notebook we use tensorflow-quantum to train a quantum neural network (QNN) model for breast cancer analysis. We use the freely available breast cancer dataset (with labels) for this purpose. To reduce complexity we have restricted our attention to the top 4 most relevant features. This was ascertained separately via 'feature selection'. After standard preprocessing we prepared two training datasets as follows:

1. 'Amplitude Encoding' approach - each feature is modeled by one qubit. After rescaling all input values are between 0 and 1. So we apply the 'rotation along y-axis' proportional to the input value to each qubit state.
2. 'Binary Encoding' approach - each feature is modeled by one qubit. After rescaling all input values are between 0 and 1. We define an intermediate theshold value and map each input value less than the threshold value to 0 and greater than the threshold value to 1. In this manner we obtain a binary dataset and then we apply the X-gate to those qubits, whose corresponding input value is one.

Finally, we adapt the QNN construction in this tutorial https://www.tensorflow.org/quantum/tutorials/mnist to our setting and compare the results of the two approaches.

```python
import pandas as pd
import tensorflow as tf
import tensorflow_quantum as tfq
import time
import cirq
import sympy
import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt
from cirq.contrib.svg import SVGCircuit
```

```python
# The breast cancer dataset can be obtained from here - https://github.com/snigdhayan/NeuralNetworks/blob/master/Ludwig/breast_cancer_dataset.csv
df = pd.read_csv('./breast_cancer_dataset.csv')
df.head()
```

      mean_radius
      mean_texture
      mean_perimeter
      mean_area
      mean_smoothness
      mean_compactness
      mean_concavity
      mean_concave_points
      mean_symmetry
      mean_fractal_dimension
      ...
      worst_texture
      worst_perimeter
      worst_area
      worst_smoothness
      worst_compactness
      worst_concavity
      worst_concave_points
      worst_symmetry
      worst_fractal_dimension
      label

      0
      17.99
      10.38
      122.80
      1001.0
      0.11840
      0.27760
      0.3001
      0.14710
      0.2419
      0.07871
      ...
      17.33
      184.60
      2019.0
      0.1622
      0.6656
      0.7119
      0.2654
      0.4601
      0.11890
      0

      1
      20.57
      17.77
      132.90
      1326.0
      0.08474
      0.07864
      0.0869
      0.07017
      0.1812
      0.05667
      ...
      23.41
      158.80
      1956.0
      0.1238
      0.1866
      0.2416
      0.1860
      0.2750
      0.08902
      0

      2
      19.69
      21.25
      130.00
      1203.0
      0.10960
      0.15990
      0.1974
      0.12790
      0.2069
      0.05999
      ...
      25.53
      152.50
      1709.0
      0.1444
      0.4245
      0.4504
      0.2430
      0.3613
      0.08758
      0

      3
      11.42
      20.38
      77.58
      386.1
      0.14250
      0.28390
      0.2414
      0.10520
      0.2597
      0.09744
      ...
      26.50
      98.87
      567.7
      0.2098
      0.8663
      0.6869
      0.2575
      0.6638
      0.17300
      0

      4
      20.29
      14.34
      135.10
      1297.0
      0.10030
      0.13280
      0.1980
      0.10430
      0.1809
      0.05883
      ...
      16.67
      152.20
      1575.0
      0.1374
      0.2050
      0.4000
      0.1625
      0.2364
      0.07678
      0

5 rows × 31 columns

```python
scaling_table = {}
for column in df:
    column_max = df[column].max()
    column_min = df[column].min()
    df[column] = (df[column] - column_min) / (column_max - column_min)
    scaling_table[column] = {'min': column_min, 'max': column_max}
df.head()
```

      mean_radius
      mean_texture
      mean_perimeter
      mean_area
      mean_smoothness
      mean_compactness
      mean_concavity
      mean_concave_points
      mean_symmetry
      mean_fractal_dimension
      ...
      worst_texture
      worst_perimeter
      worst_area
      worst_smoothness
      worst_compactness
      worst_concavity
      worst_concave_points
      worst_symmetry
      worst_fractal_dimension
      label

      0
      0.521037
      0.022658
      0.545989
      0.363733
      0.593753
      0.792037
      0.703140
      0.731113
      0.686364
      0.605518
      ...
      0.141525
      0.668310
      0.450698
      0.601136
      0.619292
      0.568610
      0.912027
      0.598462
      0.418864
      0.0

      1
      0.643144
      0.272574
      0.615783
      0.501591
      0.289880
      0.181768
      0.203608
      0.348757
      0.379798
      0.141323
      ...
      0.303571
      0.539818
      0.435214
      0.347553
      0.154563
      0.192971
      0.639175
      0.233590
      0.222878
      0.0

      2
      0.601496
      0.390260
      0.595743
      0.449417
      0.514309
      0.431017
      0.462512
      0.635686
      0.509596
      0.211247
      ...
      0.360075
      0.508442
      0.374508
      0.483590
      0.385375
      0.359744
      0.835052
      0.403706
      0.213433
      0.0

      3
      0.210090
      0.360839
      0.233501
      0.102906
      0.811321
      0.811361
      0.565604
      0.522863
      0.776263
      1.000000
      ...
      0.385928
      0.241347
      0.094008
      0.915472
      0.814012
      0.548642
      0.884880
      1.000000
      0.773711
      0.0

      4
      0.629893
      0.156578
      0.630986
      0.489290
      0.430351
      0.347893
      0.463918
      0.518390
      0.378283
      0.186816
      ...
      0.123934
      0.506948
      0.341575
      0.437364
      0.172415
      0.319489
      0.558419
      0.157500
      0.142595
      0.0

5 rows × 31 columns

```python
y_columns = df['label'].values
THRESHOLD = 0.5

worst_concave_points = df['worst_concave_points']
worst_perimeter = df['worst_perimeter']
mean_concave_points = df['mean_concave_points']
worst_radius = df['worst_radius']

worst_concave_points_bin = df['worst_concave_points'].apply(lambda x : 0 if x < THRESHOLD else 1)
worst_perimeter_bin = df['worst_perimeter'].apply(lambda x : 0 if x < THRESHOLD else 1)
mean_concave_points_bin = df['mean_concave_points'].apply(lambda x : 0 if x < THRESHOLD else 1)
worst_radius_bin = df['worst_radius'].apply(lambda x : 0 if x < THRESHOLD else 1)

sel_df = pd.concat([worst_concave_points, worst_perimeter, mean_concave_points, worst_radius], axis=1)
sel_df.rename(columns={0: 'wcp', 1: 'wp', 2: 'mcp', 3: 'wr'}, inplace=True)

sel_df_bin = pd.concat([worst_concave_points_bin, worst_perimeter_bin, mean_concave_points_bin, worst_radius_bin], axis=1)#pd.concat([mean,error,worst], axis=1)
sel_df_bin.rename(columns={0: 'wcp', 1: 'wp', 2: 'mcp', 3: 'wr'}, inplace=True)

```

```python
train_test_split = 400
y_train = y_columns.reshape(1,-1)[0]
y_test = y_train[train_test_split:]
y_train = y_train[:train_test_split]

x_train = sel_df.values
x_test = x_train[train_test_split:]
x_train = x_train[:train_test_split]

x_train_bin = sel_df_bin.values
x_test_bin = x_train_bin[train_test_split:]
x_train_bin = x_train_bin[:train_test_split]

```

```python
def convert_to_circuit(data):
    """Encode prepared data into quantum datapoint - Amplitude Encoding."""
    values = np.ndarray.flatten(data)
    qubits = cirq.GridQubit.rect(4, 1)
    circuit = cirq.Circuit()
    for i, value in enumerate(values):
        rad = (value) * np.pi
        circuit.append(cirq.ry(rad)(qubits[i]))

    return circuit
```

```python
def convert_to_circuit_bin(data):
    """Encode prepared data into quantum datapoint - Binary Encoding."""
    values = np.ndarray.flatten(data)
    qubits = cirq.GridQubit.rect(4, 1)
    circuit = cirq.Circuit()
    for i, value in enumerate(values):
        if value == 1:
            circuit.append(cirq.X(qubits[i]))

    return circuit
```

```python
x_train_circ = [convert_to_circuit(x) for x in x_train]
x_test_circ = [convert_to_circuit(x) for x in x_test]

SVGCircuit(x_train_circ[1])
```

    findfont: Font family ['Arial'] not found. Falling back to DejaVu Sans.

```python
x_train_circ_bin = [convert_to_circuit_bin(x) for x in x_train_bin]
x_test_circ_bin = [convert_to_circuit_bin(x) for x in x_test_bin]

SVGCircuit(x_train_circ_bin[1])
```

```python
x_train_tfcirc = tfq.convert_to_tensor(x_train_circ)
x_test_tfcirc = tfq.convert_to_tensor(x_test_circ)

x_train_tfcirc_bin = tfq.convert_to_tensor(x_train_circ_bin)
x_test_tfcirc_bin = tfq.convert_to_tensor(x_test_circ_bin)
```

```python
class CircuitLayerBuilder():
    def __init__(self, data_qubits, readout):
        self.data_qubits = data_qubits
        self.readout = readout

    def add_layer(self, circuit, gate, prefix):
        for i, qubit in enumerate(self.data_qubits):
            symbol = sympy.Symbol(prefix + '-' + str(i))
            circuit.append(gate(qubit, self.readout)**symbol)
```

```python
def create_quantum_model():
    """Create a QNN model circuit and readout operation to go along with it."""
    data_qubits = cirq.GridQubit.rect(4, 1)  # a 4x4 grid.
    readout = cirq.GridQubit(-1, -1)         # a single qubit at [-1,-1]
    circuit = cirq.Circuit()

    # Prepare the readout qubit.
    circuit.append(cirq.X(readout))
    circuit.append(cirq.H(readout))

    builder = CircuitLayerBuilder(
        data_qubits = data_qubits,
        readout=readout)

    # Then add layers (experiment by adding more).
    builder.add_layer(circuit, cirq.XX, "xx1")
    builder.add_layer(circuit, cirq.ZZ, "zz1")

    # Finally, prepare the readout qubit.
    circuit.append(cirq.H(readout))

    return circuit, cirq.Z(readout)
```

```python
model_circuit, model_readout = create_quantum_model()
```

```python
SVGCircuit(model_circuit)
```

```python
# Build the Keras model.
qnn_model = tf.keras.Sequential([
    # The input is the data-circuit, encoded as a tf.string
    tf.keras.layers.Input(shape=(), dtype=tf.string),
    # The PQC layer returns the expected value of the readout gate, range [-1,1].
    tfq.layers.PQC(model_circuit, model_readout),
])
```

```python
# Build the Keras model.
qnn_model_bin = tf.keras.Sequential([
    # The input is the data-circuit, encoded as a tf.string
    tf.keras.layers.Input(shape=(), dtype=tf.string),
    # The PQC layer returns the expected value of the readout gate, range [-1,1].
    tfq.layers.PQC(model_circuit, model_readout),
])
```

```python
y_train_hinge = 2.0*y_train-1.0
y_test_hinge = 2.0*y_test-1.0
```

```python
def hinge_accuracy(y_true, y_pred):
    y_true = tf.squeeze(y_true) > 0.0
    y_pred = tf.squeeze(y_pred) > 0.0
    result = tf.cast(y_true == y_pred, tf.float32)

    return tf.reduce_mean(result)
```

```python
qnn_model.compile(
    loss=tf.keras.losses.Hinge(),
    optimizer=tf.keras.optimizers.Adam(),
    metrics=[hinge_accuracy])
qnn_model.summary()
```

    Model: "sequential"
    _________________________________________________________________
    Layer (type)                 Output Shape              Param #
    =================================================================
    pqc (PQC)                    (None, 1)                 8
    =================================================================
    Total params: 8
    Trainable params: 8
    Non-trainable params: 0
    _________________________________________________________________

```python
qnn_model_bin.compile(
    loss=tf.keras.losses.Hinge(),
    optimizer=tf.keras.optimizers.Adam(),
    metrics=[hinge_accuracy])
qnn_model_bin.summary()
```

    Model: "sequential_1"
    _________________________________________________________________
    Layer (type)                 Output Shape              Param #
    =================================================================
    pqc_1 (PQC)                  (None, 1)                 8
    =================================================================
    Total params: 8
    Trainable params: 8
    Non-trainable params: 0
    _________________________________________________________________

```python
EPOCHS = 5
BATCH_SIZE = 8

NUM_EXAMPLES = len(x_train_tfcirc)
```

```python
x_train_tfcirc_sub = x_train_tfcirc[:NUM_EXAMPLES]
y_train_hinge_sub = y_train_hinge[:NUM_EXAMPLES]
```

```python
x_train_tfcirc_sub_bin = x_train_tfcirc_bin[:NUM_EXAMPLES]
y_train_hinge_sub_bin = y_train_hinge[:NUM_EXAMPLES]
```

```python
start_time = time.time()
qnn_history = qnn_model.fit(
      x_train_tfcirc_sub, y_train_hinge_sub,
      batch_size=BATCH_SIZE,
      epochs=EPOCHS,
      verbose=1,
      validation_data=(x_test_tfcirc, y_test_hinge))

qnn_results = qnn_model.evaluate(x_test_tfcirc, y_test)
qnn_training_time = time.time() - start_time
```

    Epoch 1/5
    50/50 [==============================] - 1s 11ms/step - loss: 0.9960 - hinge_accuracy: 0.5925 - val_loss: 0.9763 - val_hinge_accuracy: 0.8295
    Epoch 2/5
    50/50 [==============================] - 0s 7ms/step - loss: 0.9601 - hinge_accuracy: 0.7025 - val_loss: 0.9148 - val_hinge_accuracy: 0.8409
    Epoch 3/5
    50/50 [==============================] - 0s 7ms/step - loss: 0.8981 - hinge_accuracy: 0.7200 - val_loss: 0.8285 - val_hinge_accuracy: 0.8636
    Epoch 4/5
    50/50 [==============================] - 0s 7ms/step - loss: 0.8269 - hinge_accuracy: 0.7425 - val_loss: 0.7463 - val_hinge_accuracy: 0.8864
    Epoch 5/5
    50/50 [==============================] - 0s 7ms/step - loss: 0.7688 - hinge_accuracy: 0.7650 - val_loss: 0.6879 - val_hinge_accuracy: 0.8920
    6/6 [==============================] - 0s 8ms/step - loss: 0.6879 - hinge_accuracy: 0.8478

```python
start_time = time.time()
qnn_history_bin = qnn_model_bin.fit(
      x_train_tfcirc_sub_bin, y_train_hinge_sub_bin,
      batch_size=BATCH_SIZE,
      epochs=EPOCHS,
      verbose=1,
      validation_data=(x_test_tfcirc_bin, y_test_hinge))

qnn_results_bin = qnn_model.evaluate(x_test_tfcirc_bin, y_test)
qnn_training_time_bin = time.time() - start_time
```

    Epoch 1/5
    50/50 [==============================] - 0s 10ms/step - loss: 1.1050 - hinge_accuracy: 0.2900 - val_loss: 1.1018 - val_hinge_accuracy: 0.1364
    Epoch 2/5
    50/50 [==============================] - 0s 7ms/step - loss: 0.9665 - hinge_accuracy: 0.4925 - val_loss: 0.8628 - val_hinge_accuracy: 0.8239
    Epoch 3/5
    50/50 [==============================] - 0s 6ms/step - loss: 0.8313 - hinge_accuracy: 0.6900 - val_loss: 0.6369 - val_hinge_accuracy: 0.8239
    Epoch 4/5
    50/50 [==============================] - 0s 6ms/step - loss: 0.7304 - hinge_accuracy: 0.6900 - val_loss: 0.4931 - val_hinge_accuracy: 0.8239
    Epoch 5/5
    50/50 [==============================] - 0s 7ms/step - loss: 0.6753 - hinge_accuracy: 0.6900 - val_loss: 0.4169 - val_hinge_accuracy: 0.8239
    6/6 [==============================] - 0s 7ms/step - loss: 0.3845 - hinge_accuracy: 0.8241

```python
qnn_accuracy = qnn_results[1]
qnn_accuracy_bin = qnn_results_bin[1]
sns.barplot(x=['Amplitude Encoding','Binary Encoding'],y=[qnn_accuracy,qnn_accuracy_bin])
plt.title('Hinge Accuracy Score')
```

```python
amplitude_training_time = round(qnn_training_time,2)
binary_training_time = round(qnn_training_time_bin,2)

sns.barplot(x=['Amplitude Encoding','Binary Encoding'],y=[amplitude_training_time,binary_training_time])
plt.title('Training Time (in seconds)')
```

```python
plt.plot(qnn_history.history['val_loss'], label='Amplitude Encoding')
plt.plot(qnn_history_bin.history['val_loss'], label='Binary Encoding')
plt.title('Validation Loss')
plt.legend(('Amplitude Encoding', 'Binary Encoding'))
```

```python
plt.plot(qnn_history.history['val_hinge_accuracy'])
plt.plot(qnn_history_bin.history['val_hinge_accuracy'])
plt.title('Validation Hinge Accuracy')
plt.legend(('Amplitude Encoding', 'Binary Encoding'))
```


## PDE
The TTI wave equation for the square slowness m, thomsen parametersd $\epsilon$, $\delta$, $\theta$ and $\Phi$ (dip and asymuth)  and a source q is given in 3D by :

\begin{cases}
 &m \frac{d^2 p(x,t)}{dt^2} - (1+2\epsilon)(G_{\bar{x}\bar{x}} + G_{\bar{y}\bar{y}}) p(x,t) - \sqrt{(1+2\delta)}G_{\bar{z}\bar{z}} r(x,t) =q  \\
 &m \frac{d^2 r(x,t)}{dt^2} -  \sqrt{(1+2\delta)}(G_{\bar{x}\bar{x}} + G_{\bar{y}\bar{y}}) p(x,t) - G_{\bar{z}\bar{z}} r(x,t) =q    \\
 &u(.,0) = 0 \\
 &\frac{d u(x,t)}{dt}|_{t=0} = 0
\end{cases}

where
\begin{cases}
G_{\bar{x}\bar{x}} & = cos(\phi)^2 cos(\theta)^2 \frac{d^2}{dx^2} +sin(\phi)^2 cos(\theta)^2 \frac{d^2}{dy^2}+ sin(\theta)^2 \frac{d^2}{dz^2} + sin(2\phi) cos(\theta)^2 \frac{d^2}{dx dy} - sin(\phi) sin(2\theta) \frac{d^2}{dy dz} -cos(\phi) sin(2\theta) \frac{d^2}{dx dz} \\
G_{\bar{y}\bar{y}} & =  sin(\phi)^2 \frac{d^2}{dx^2} +cos(\phi)^2  \frac{d^2}{dy^2} - sin(2\phi)^2 \frac{d^2}{dx dy}\\
G_{\bar{z}\bar{z}} & = cos(\phi)^2 sin(\theta)^2 \frac{d^2}{dx^2} +sin(\phi)^2 sin(\theta)^2 \frac{d^2}{dy^2}+ cos(\theta)^2 \frac{d^2}{dz^2} + sin(2\phi) sin(\theta)^2 \frac{d^2}{dx dy} + sin(\phi) sin(2\theta) \frac{d^2}{dy dz} +cos(\phi) sin(2\theta) \frac{d^2}{dx dz} \\
\end{cases}
with the zero initial conditons to guaranty unicity of the solution

It correspondto a velocity model where the velocity in the z direction is different from the velocity in the xy plane

```python
from sympy import *
from sympy.abc import *
from sympy.galgebra.ga import *
import numpy as np
from numpy import linalg as LA
from __future__ import print_function
from functools import reduce
from operator import mul
init_printing()
```

```python
p=Function('p')
r=Function('r')

s,h,x,y,z = symbols('s h x y z')
m=M(x,y,z)
q=Q(x,y,z,t)
d=D(x,y,z,t)
e=E(x,y,z)

A=epsilon(x,y,z)  # (1 + 2epsilon) but make the symbolic simpler
B=delta(x,y,z) # sqrt(1 + 2epsilon) but make the symbolic simpler
Th=theta(x,y,z)
Ph=phi(x,y,z)

```

```python
order=1

indxx = [(x + i * h) for i in range(-order, order + 1)]
indyy = [(y + i * h) for i in range(-order, order + 1)]
indzz = [(z + i * h) for i in range(-order, order + 1)]
```

```python
dttp=as_finite_diff(p(x,y,z,t).diff(t,t), [t-s,t, t+s])
dttr=as_finite_diff(r(x,y,z,t).diff(t,t), [t-s,t, t+s])
dtp=as_finite_diff(p(x,y,z,t).diff(t), [t-s,t])
dtr=as_finite_diff(r(x,y,z,t).diff(t), [t-s,t])
# Spacial finite differences can easily be extended to higher order by increasing the list of sampling point in the next expression.
# Be sure to keep this stencil symmetric and everything else in the notebook will follow.
dxxp=as_finite_diff(p(x,y,z,t).diff(x,x), indxx)
dyyp=as_finite_diff(p(x,y,z,t).diff(y,y), indyy)
dzzp=as_finite_diff(p(x,y,z,t).diff(z,z), indzz)
dxxr=as_finite_diff(r(x,y,z,t).diff(x,x), indxx)
dyyr=as_finite_diff(r(x,y,z,t).diff(y,y), indyy)
dzzr=as_finite_diff(r(x,y,z,t).diff(z,z), indzz)

# My 4th order stencil for cross derivatives
dxzp = .5/(h**2)*(-2*p(x,y,z,t) + p(x,y,z+h,t) + p(x,y,z-h,t) - p(x+h,y,z-h,t) + p(x-h,y,z,t) - p(x-h,y,z+h,t) + p(x+h,y,z,t))
dxzr = .5/(h**2)*(-2*r(x,y,z,t) + r(x,y,z+h,t) + r(x,y,z-h,t) - r(x+h,y,z-h,t) + r(x-h,y,z,t) - r(x-h,y,z+h,t) + r(x+h,y,z,t))

dxyp = .5/(h**2)*(-2*p(x,y,z,t) + p(x,y+h,z,t) + p(x,y-h,z,t) - p(x+h,y-h,z,t) + p(x-h,y,z,t) - p(x-h,y+h,z,t) + p(x+h,y,z,t))
dxyr = .5/(h**2)*(-2*r(x,y,z,t) + r(x,y+h,z,t) + r(x,y-h,z,t) - r(x+h,y-h,z,t) + r(x-h,y,z,t) - r(x-h,y+h,z,t) + r(x+h,y,z,t))

dyzp = .5/(h**2)*(-2*p(x,y,z,t) + p(x,y,z+h,t) + p(x,y,z-h,t) - p(x,y+h,z-h,t) + p(x,y-h,z,t) - p(x,y-h,z+h,t) + p(x,y+h,z,t))
dyzr = .5/(h**2)*(-2*r(x,y,z,t) + r(x,y,z+h,t) + r(x,y,z-h,t) - r(x,y+h,z-h,t) + r(x,y-h,z,t) - r(x,y-h,z+h,t) + r(x,y+h,z,t))
```

```python
# add the *arg input and done
def cross_deriv(*args, **kwargs):
    deriv=0
    order = kwargs.get('order', 1)
    dims = kwargs.get('dims', (x, y))
    diff = kwargs.get('diff', h)
    assert(isinstance(dims, tuple) and len(dims) == 2)
    ind1r = [(dims[0] + i * diff) for i in range(-int((order) / 2) + 1 - (order<4), int((order + 1) / 2) + 2 - (order<4))]
    ind2r = [(dims[1] + i * diff) for i in range(-int((order) / 2) + 1 - (order<4), int((order + 1) / 2) + 2 - (order<4))]
    ind1l = [(dims[0] - i * diff) for i in range(-int((order) / 2) + 1 - (order<4), int((order + 1) / 2) + 2 - (order<4))]
    ind2l = [(dims[1] - i * diff) for i in range(-int((order) / 2) + 1 - (order<4), int((order + 1) / 2) + 2 - (order<4))]
    cx = finite_diff_weights(1, ind1r, dims[0])
    cx = cx[-1][-1]
    cy = finite_diff_weights(1, ind2r, dims[1])
    cy = cy[-1][-1]
    for i in range(0,len(ind1r)):
        for j in range(0,len(ind2r)):
            var1 = [a.subs({dims[0]: ind1r[i], dims[1]: ind2r[j]}) for a in args]
            var2 = [a.subs({dims[0]: ind1l[i], dims[1]: ind2l[j]}) for a in args]
            deriv += .25 * cy[j] * cx[i] * reduce(mul, var1, 1) + .25 * cy[len(ind2l)-j-1] * cx[len(ind1l)-i-1] * reduce(mul, var2, 1)
    return deriv
```

```python
cross_deriv(p(x, y, z, t), order=1, dims=(x,z))
```

```python
Gxxp = c**2 * a**2 * dxxp + d**2 * a**2 * dyyp + b**2 * dzzp + 2 * d * c * a**2 * dxyp - d * 2 * b * a * dyzp - c * 2 * b * a * dxzp
Gyyp = b**2 * dxxp + c**2 * dyyp - (2 * d * c)**2 * dxyp
Gzzr = c**2 * b**2 * dxxr + d**2 * b**2 * dyyr + a**2 * dzzr + 2 * d * c * b**2 * dxyr + d * 2 * b * a * dyzr + c * 2 * b * a * dxzr
```

```python
def Bhaskarasin(angle):
    return 16 * angle * (3.14 - abs(angle))/(49.34 - 4 * abs(angle) * (3.14 - abs(angle)))

def Bhaskaracos(angle):
    return Bhaskarasin(angle + 1.57)

a = Bhaskaracos(Th)
b = Bhaskarasin(Th)
c = Bhaskaracos(Ph)
d = Bhaskaracos(Ph)
```

```python
print(a)
print(b)
print(c)
print(d)
```

```python
stencilp = 2 * s**2 / (2 * m + s * damp) * (2 * m / s**2 * p(x, y, z, t) + (s * damp - 2 * m) / (2 * s**2) * p(x, y, z, t-s) + A * (Gxxp + Gyyp) + B * Gzzr)
```

```python
#stencilp = simplify(expand(stencilp))
```

```python
#cse((stencilp))
```

```python
stencilr =  2 * s**2 / (2 * m + s * damp) * (2 * m / s**2 * r(x, y, z, t) + (s * damp - 2 * m) / (2 * s**2) * r(x, y, z, t-s) + B * (Gxxp + Gyyp) + Gzzr)
```

```python
#stencilr=factor(simplify(expand(stencilr)))
```

```python
cse((stencilp,stencilr))
```

```python

```


# Week 5 Programming Assignment

Remark:

Please upload your solutions of this assignment to Canvas with a file named "Programming_Assignment_5 _yourname.ipynb" before deadline.

=================================================================================================================

## Problem 1 : Use stochastic gradient descent method to train CIFAR10 with ResNet18.

Apply the following two different learning rate schedules respectively

(1) run 120 epochs with a fixed learning rate 0.1,

(2) run 120 epochs, and divide the learning rate by 10 every 30 epochs, which can achieve 94% test accuracy.

and print the results with the following format:

   "Epoch: i, Learning rate: lr$_i$, Training accuracy: $a_i$, Test accuracy: $b_i$"

where $i=1,2,3,...$ means the $i$-th epoch,  $a_i$ and $b_i$ are the training accuracy and test accuracy computed at the end of $i$-th epoch, and lr$_i$ is the learning rate of $i$-th epoch.

Optional Problem: Try to find some other learning rate schedules to achieve $94$% or higher test accuracy with less epochs.

```python
# Write your code to solve Problem 1.

```

=================================================================================================================

## Problem 2 :  Consider the possion equation
\begin{equation}\label{1Dposi}
\left\{
\begin{aligned}
-u''&= f, \,\, 0<x<1, \\
 u(0)&=u(1)=0.
\end{aligned}
\right.
\end{equation}

Assume $f(x)=1$, then the exact solution is $u(x)=\frac{1}{2}x(1-x)$. Given the partition with the grid points
$x_i=\frac{i}{n+1}, i=0,1,\cdots,n+1$, then by finite element discretization,
we obtain

\begin{equation}\label{matrix}
A\ast \mu =b,~~ A=\frac{1}{h}[-1,2,-1].
\end{equation}

Use gradient descent method to solve the above problem with random initial guess $\mu^0$:

$$
\mu^{m} = \mu^{m-1} - \eta ( A* \mu^{m-1}- b),~~ m=1,2,3,...,M
$$

Set $n=15$ and $M=10$.

(1) Plot the curves of $u$ and $\mu^{M}$.

(2) Compute the error of the residual by $e^m = \sqrt{\sum_{i=0}^{n+1}|(A* \mu^{m}- b)_i |^2},~~ m=1,2,3,...,M$ and the index $i$ means the $i$-th entry of the vector. Plot a curve, where x-axis is $m=1,2,3,...,M$ and y-axis is $e^m$.

(3) Find the minumum $M$ when  $e^M = \sqrt{\sum_{i=0}^{n+1}|(A* \mu^{M}- b)_i |^2}<10^{-4}$ and record the computational time cost.

```python
import numpy as np
import time
import matplotlib.pyplot as plt

######## FEM_GD  ########
# Write your code in FEM_GD function to compute one gradient descent iteration
def FEM_GD():

######## parameter definition ########
J = 4                                # grid level
n = 2**J - 1                         # number of inner grid points
h = 1/ 2**J                          # length of grid interval
x = np.arange(1, n + 1) *h           # grid points
u = 1/2*x*(1-x)                      # true solution at grid points
b = np.ones(n)*h                     # right-hand-size term
u1 = (np.random.rand(n)*2-1+np.sin(4*np.pi*x))/2  # initial value for u
M = 10
t0 = time.time()                     # initial time

######## compute numerical solution ########
err = []                             # create a list to save the error of each iteration
for m in range(M):
  u1 = FEM_GD({Define a FEM_GD function to compute one gradient descent iteration})
  temp=np.array([u1[0]*A[1]+u1[1]*A[2]])
  for j in np.arange(1,len(u1)-1):
      temp=np.append(temp,np.dot(u1[j-1:j+2],A))
  Au=np.append(temp,u1[-2]*A[0]+u1[-1]*A[1])
  err.append(np.linalg.norm(Au-b))   # compute the error of m-th iteration and save it to the list
print('time cost', time.time() - t0)
######## plot the exact solution and numerical solution ########
plt.figure()
plt.title('Exact solution and numerical solution')
plot = plt.plot(x,u,label='Exact solution')
plot = plt.plot(x,u1,'*',label='Numerical solution')
plt.legend()
plt.show()

######## plot the l2 norm of the error vs iterations ########
plt.figure()
plt.title('Error vs number of iterations using FEM and gradient descent')
plot = plt.plot(err)
plt.xlabel('Number of iterations')
plt.yscale('log')
plt.ylabel('Error')
plt.show()
```

=================================================================================================================

## Problem 3 : Consider the Poisson equation described in Problem 1, call the Multigrid code given in the following cell to obtain a solution $u^{\nu}$.
Use multigrid method to solve the above problem with random initial guess $\mu^0$:

$$
\mu^{m} =  MG1(\mu^{m-1}),~~ m=1,2,3,...,M
$$

Set $n=15$ and $M=10$.

(1) Plot the curves of $u$ and $\mu^{M}$ and record the computational time cost.

(2) Compute the error of the residual by $e^m = \sqrt{\sum_{i=0}^{n+1}|(A* \mu^{m}- b)_i |^2},~~ m=1,2,3,...,M$ and the index $i$ means the $i$-th entry of the vector. Plot a curve, where x-axis is $m=1,2,3,...,M$ and y-axis is $e^m$.

```python
import numpy as np
import time
import matplotlib.pyplot as plt

######## MG1 definition ########
def MG1(b, u0, J, v):  # \mu=[\mu_1,\mu_2,\mu_3,...,\mu_J]
  if len(b)!=len(u0):
    print('input size not equal')
  if len(v)!=J:
    print('length of v not equal to J')
  B=[0,b]
  U=[0,u0]
  R=np.array([1/2,1,1/2])
  for l in np.arange(1,J+1):
    h_l=1/2**(J+1-l)
    A=np.array([-1,2,-1]/h_l)
    if l<J:
      for i in np.arange(0,v[l-1]):
        temp=np.array([U[l][0]*A[1]+U[l][1]*A[2]])
        for j in np.arange(1,len(U[l])-1):
            temp=np.append(temp,np.dot(U[l][j-1:j+2],A))
        temp=np.append(temp,U[l][-2]*A[0]+U[l][-1]*A[1])
        U[l]+=1/4*h_l*(B[l]-temp)
      U.append(np.zeros((len(U[l])-1)//2))
      newb=[]
      temp=np.array([U[l][0]*A[1]+U[l][1]*A[2]])
      for j in np.arange(1,len(U[l])-1):
          temp=np.append(temp,np.dot(U[l][j-1:j+2],A))
      temp=np.append(temp,U[l][-2]*A[0]+U[l][-1]*A[1])
      for k in range((len(U[l])-1)//2):
        newb.append(np.dot((B[l]-temp)[2*k:2*k+3],R))
      B.append(newb)
    else:
      for i in np.arange(0,v[l-1]):
        temp=np.array(U[l][0]*A[1])
        U[l]+=1/4*h_l*(B[l]-temp)

  for l in np.arange(J-1,0,-1):
    temp=[1/2*U[l+1][0]]
    for i in np.arange(1,len(U[l+1])*2):
      if i%2==1:
        temp.append(U[l+1][(i-1)//2])
      else:
        temp.append(1/2*(U[l+1][(i-2)//2]+U[l+1][i//2]))
    temp.append(1/2*U[l+1][-1])
    U[l]+=temp
  return U[1]

######## parameter definition ########
J = 4                                # grid level
n = 2**J - 1                         # number of inner grid points
h = 1/ 2**J                          # length of grid interval
x = np.arange(1, n + 1) *h           # grid points
u = 1/2*x*(1-x)                      # true solution at grid points
b = np.ones(n)*h                     # right-hand-size term
u1 = np.random.rand(n)*2-1           # initial value for u
M = 10
t0 = time.time()

######## compute numerical solution ########
err = []                             # create a list to save the error of each iteration
{Write your code here to call MG1 function}
print('time cost', time.time() - t0)
######## plot the exact solution and numerical solution ########
plt.figure()
plt.title('Exact solution and numerical solution')
plot = plt.plot(x,u,label='Exact solution')
plot = plt.plot(x,u1,'*',label='Numerical solution')
plt.legend()
plt.show()

######## plot the l2 norm of the error vs iterations ########
plt.figure()
plt.title('Error vs number of iterations using multigrid')
plot = plt.plot(err)
plt.xlabel('Number of iterations')
plt.yscale('log')
plt.ylabel('Error')
plt.show()
```

=================================================================================================================


# Analiza i projektiranje računalom - 4. domaća zadaća

U okviru ove zadaće treba izraditi programsku implementaciju genetskog (evolucijskog) algoritma koja će pronalaziti minimum zadane višedimenzijske funkcije. Potrebno je omogućiti da se bez prevođenja programa može definirati:

- prikaz rješenja koji će algoritam koristiti
- veličina populacije
- vjerojatnost mutacije (i križanja, ako algoritam koristi taj parametar)
- broj evaluacija funkcije cilja kao kriterij zaustavljanja (ostali su proizvoljni)

Programsko ostvarenje mora podržavati *binarni prikaz rješenja* te *prikaz brojem s pomičnom točkom* (način prikaza je parametar algoritma). Oba prikaza trebaju definirati donju i gornju granicu domene (eksplicitna ograničenja) i broj varijabli. Binarni prikaz dodatno treba definirati željenu preciznost (npr. kao broj decimala ili najveći dopušteni razmak susjednih rješenja). Za svaki od prikaza potrebno je definirati barem do *dva operatora križanja* i barem po *jedan operator mutacije*. Svi operatori moraju podržavati definirana eksplicitna ograničenja.

**Ispis algoritma** je proizvoljan, no mora omogućiti uvid u ove podatke:

- broj evaluacija
- trenutna najbolja jedinka
- vrijednost funkcije cilja najbolje jedinke

Preporučena (ali nipošto i jedina moguća) inačica GA je 3-turnirski eliminacijski odabir:

```pseudocode
stvori populaciju
evaluiraj populaciju    //svaka jedinka ima pridruženu dobrotu

ponavljaj
    odaberi slučajno 3 jedinke
    izbaci najlošiju od 3 odabrane iz populacije

    nova jedinka = križanje(preostale dvije)
    obavi mutaciju na novoj jedinki uz neku vjerojatnost

    evaluiraj novu jedinku
    dodaj novu jedinku u populaciju
dok nije zadovoljen uvjet zaustavljanja
```

---

## Funkcije cilja

### Funkcija 1 - Rosenbrockova "banana" funkcija

$$
\begin{equation}
    f_1 \left( \vec{x} \right) = 100 \left( x_1 - {x_0}^2 \right)^2 + \left( 1 - x_0 \right)^2
\end{equation}
$$

$$
\begin{equation}
    \vec{x_{min}} =
    \begin{bmatrix}
        1 & 1
    \end{bmatrix}
\end{equation}
$$

$$
\begin{equation}
    f_1 \left( \vec{x_{min}} \right) = 0
\end{equation}
$$

### Funkcija 2

$$
\begin{equation}
    f_3 \left( \vec{x} \right) = \sum_i{\left( x_i - i \right)^2}
\end{equation}
$$

$$
\begin{equation}
    \vec{x_{min}} =
    \begin{bmatrix}
        1 & 2 & 3 & \dots & n
    \end{bmatrix}
\end{equation}
$$

$$
\begin{equation}
    f_3 \left( \vec{x_{min}} \right) = 0
\end{equation}
$$

### Funkcija 3 - Schafferova F6 funkcija

$$
\begin{equation}
    f_6 \left( \vec{x} \right) = 0.5 + \frac{%
        \sin^2 \left( \sqrt{\sum{{x_i}^2}} \right) - 0.5
    }{%
        \left( 1 + 0.001 \sum{{x_i}^2}\right)^2
    }
\end{equation}
$$

$$
\begin{equation}
    \vec{x_{min}} = \vec{0}
\end{equation}
$$

$$
\begin{equation}
    f_6 \left( \vec{x_{min}} \right) = 0
\end{equation}
$$

### Funkcija 4 - Schafferova F7 funkcija

$$
\begin{equation}
    f_7 \left( \vec{x} \right) = \left( \sum{{x_i}^2} \right)^{\frac{1}{4}} \cdot
    \left(
        1 + \sin^2 \left(
            50 \left(
                \sum{x_i}^2
            \right)^{\frac{1}{10}}
        \right)
    \right)
\end{equation}
$$

$$
\begin{equation}
    \vec{x_0} =
    \begin{bmatrix}
        0 & 0
    \end{bmatrix}
\end{equation}
$$

$$
\begin{equation}
    f_4 \left( \vec{x_{min}} \right) = 0
\end{equation}
$$

## Priprema za izvođenje

```python
import os

CD_KEY = "--HW04_IN_ROOT"
```

```python
if (
    CD_KEY not in os.environ
    or os.environ[CD_KEY] is None
    or len(os.environ[CD_KEY]) == 0
    or os.environ[CD_KEY] == "false"
):
    %cd ..
else:
    print(os.getcwd())

os.environ[CD_KEY] = "true"
```

    /mnt/data/projekti/faks/AIPR/dz/dz-04

## Učitavanje paketa

```python
import copy
import os
import pickle
import sys
import warnings

import array_to_latex as a2l
from IPython.display import display, Markdown
from matplotlib import pyplot as plt
import numpy as np
from tqdm import tqdm

from src.evolution.crossover import *
from src.evolution.encoding import *
from src.evolution.function import Function
from src.evolution.mutation import *
from src.evolution.selection import *
from src.evolutionary_tournament import EvolutionaryTournament
from src.ga import simple_ga
from src.utils import *
```

```python
np.set_printoptions(precision=2, suppress=True)
np.random.seed(21051208)
warnings.filterwarnings('ignore')
```

## Definicija pomoćnih funkcija

```python
def array_to_latex(array):
    return a2l.to_ltx(
        array,
        frmt="{:g}",
        arraytype="bmatrix",
        print_out=False
    )
```

## Definicija funkcija i početnih točaka

```python
def f1(x: np.ndarray):
    return 100 * np.square(np.square(x[1] - x[0])) + np.square(1. - x[0])
```

```python
def get_f3(length: int):
    _subtractor = np.array(range(1, length + 1))

    def _f(x: np.ndarray):
        return np.sum(np.square(x - _subtractor))

    return _f
```

```python
def f6(x: np.ndarray):
    sum_of_squares = np.sum(np.square(x))

    return (
        0.5
        + (
            (np.square(np.sin(np.sqrt(sum_of_squares))) - 0.5) /
            np.square(1 + 0.001 * sum_of_squares)
        )
    )
```

```python
def f7(x: np.ndarray):
    sum_of_squares = np.sum(np.square(x))

    return (
        np.power(sum_of_squares, 0.25) * (
            1.0 + np.square(
                np.sin(
                    50 * np.power(sum_of_squares, 0.1)
                )
            )
        )
    )
```

## Zadatci

### Zadatak 1

Isprobajte vašu implementaciju GA nad svim funkcijama uz granice $\left[ -50, 150 \right]$ za sve varijable te za oba prikaza rješenja (binarni, s pomičnom točkom).

```python
t1_interval = (-50, 150)
```

Za binarni prikaz zadajte preciznost od barem tri decimalna mjesta.

```python
t1_n_decimals = 3
t1_dim = get_binary_dim_for_interval_and_decimals(
    interval=t1_interval, n_decimals=t1_n_decimals
)
display(
    Markdown(
        f"Za interval $[{t1_interval[0]}, {t1_interval[1]}]$ "
        f"trebat ćemo ${t1_dim}$ binarnih znamenaka da bi "
        f"imali preciznost na ${t1_n_decimals}$ decimale."
    )
)
```

Za interval $[-50, 150]$ trebat ćemo $18$ binarnih znamenaka da bi imali preciznost na $3$ decimale.

```python
t1_be = BinaryEncoder(
    dim=t1_dim, interval=t1_interval
)
t1_bd = BinaryDecoder(
    dim=t1_dim, interval=t1_interval
)
```

Za funkciju $f_3$ odaberite barem $5$ varijabli, a za $f_6$ i $f_7$ dvije varijable.

```python
t1_f1_n = 2
t1_f3_n = 5
t1_f6_n = 2
t1_f7_n = 2
```

```python
t1_f1 = copy.deepcopy(f6)
t1_f3 = get_f3(t1_f3_n)
t1_f6 = copy.deepcopy(f6)
t1_f7 = copy.deepcopy(f7)
```

```python
t1_f1_binary = Function(lambda x: -t1_f1(t1_bd(x)))
t1_f1_float = Function(lambda x: -t1_f1(x))
```

```python
t1_f3_binary = Function(lambda x: -t1_f3(t1_bd(x)))
t1_f3_float = Function(lambda x: -t1_f3(x))
```

```python
t1_f6_binary = Function(lambda x: -t1_f6(t1_bd(x)))
t1_f6_float = Function(lambda x: -t1_f6(x))
```

```python
t1_f7_binary = Function(lambda x: -t1_f7(t1_bd(x)))
t1_f7_float = Function(lambda x: -t1_f7(x))
```

Za sve funkcije možete smatrati da je rješenje pronađeno ako je krajnja vrijednost funkcije cilja manja od $10^{-6}$. Za neke funkcije algoritam će biti potrebno pokrenuti nekoliko puta.

**Odgovor**

```python
t1_max_retries = 3
t1_max_iterations = 10000
t1_capacity = 20
t1_elitism = 1
t1_epsilon = 1e-6
```

```python
t1_mp_schedule = lambda x: 0.05 * (np.sin(x/1000) + 1)
```

```python
t1_et_binary = EvolutionaryTournament(
    selection=TournamentSelection(),
    crossover=ANDCrossover(),
    mutation=StochasticCorruptionMutation()
)
t1_et_float = EvolutionaryTournament(
    selection=TournamentSelection(),
    crossover=ChooseOneCrossover(interval=t1_interval),
    mutation=GaussianMutation()
)
```

---

#### Funkcija $f_1$ - binarno

```python
t1_f1_binary_results = simple_ga(
    wellness_function=t1_f1_binary,
    specimen_length=t1_f1_n,
    interval=t1_interval,
    evolutionary_tournament=t1_et_binary,
    max_retries=t1_max_retries,
    max_iterations=t1_max_iterations,
    population_capacity=t1_capacity,
    elitism=t1_elitism,
    epsilon=t1_epsilon,
    mutation_scheduler=t1_mp_schedule,
    encoding_function=t1_be,
    decoding_function=t1_bd,
)
```

    [105] [0. 0.] (-0.00):   1%|          | 105/10000 [00:00<00:07, 1374.93it/s]

```python
t1_f1_binary_best = sorted(t1_f1_binary_results, key=lambda x: x.wellness)[-1]

display(
    Markdown(
        f"Pronađen je minimum vrijednosti ${-t1_f1_binary_best.wellness:.06f}$ "
        f"u točki {array_to_latex(t1_bd(t1_f1_binary_best.element))}"
    )
)
```

Pronađen je minimum vrijednosti $0.000000$ u točki \begin{bmatrix}
  0.000190736 &  0.000190736
\end{bmatrix}

#### Funkcija $f_3$ - binarno

```python
t1_f3_binary_results = simple_ga(
    wellness_function=t1_f3_binary,
    specimen_length=t1_f3_n,
    interval=t1_interval,
    evolutionary_tournament=t1_et_binary,
    max_retries=t1_max_retries,
    max_iterations=t1_max_iterations,
    population_capacity=t1_capacity,
    elitism=t1_elitism,
    epsilon=t1_epsilon,
    mutation_scheduler=t1_mp_schedule,
    encoding_function=t1_be,
    decoding_function=t1_bd,
)
```

    [9138] [0.98 1.96 2.93 3.91 6.25] (-1.58): 100%|██████████| 10000/10000 [00:05<00:00, 1922.04it/s]
    [9928] [ 0.78 -0.39  2.34  6.25  6.25] (-12.82): 100%|██████████| 10000/10000 [00:05<00:00, 1978.42it/s]
    [6046] [0.2  1.95 3.13 3.92 4.69] (-0.77): 100%|██████████| 10000/10000 [00:05<00:00, 1991.12it/s]

```python
t1_f3_binary_best = sorted(t1_f3_binary_results, key=lambda x: x.wellness)[-1]

display(
    Markdown(
        f"Pronađen je minimum vrijednosti ${-t1_f3_binary_best.wellness:.06f}$ "
        f"u točki {array_to_latex(t1_bd(t1_f3_binary_best.element))}"
    )
)
```

Pronađen je minimum vrijednosti $0.769210$ u točki \begin{bmatrix}
  0.195504 &  1.95332 &  3.1252 &  3.91866 &  4.68771
\end{bmatrix}

#### Funkcija $f_6$ - binarno

```python
t1_f6_binary_results = simple_ga(
    wellness_function=t1_f6_binary,
    specimen_length=t1_f6_n,
    interval=t1_interval,
    evolutionary_tournament=t1_et_binary,
    max_retries=t1_max_retries,
    max_iterations=t1_max_iterations,
    population_capacity=t1_capacity,
    elitism=t1_elitism,
    epsilon=t1_epsilon,
    mutation_scheduler=t1_mp_schedule,
    encoding_function=t1_be,
    decoding_function=t1_bd,
)
```

    [152] [0. 0.] (-0.00):   2%|▏         | 152/10000 [00:00<00:05, 1894.36it/s]

```python
t1_f6_binary_best = sorted(t1_f6_binary_results, key=lambda x: x.wellness)[-1]

display(
    Markdown(
        f"Pronađen je minimum vrijednosti ${-t1_f6_binary_best.wellness:.06f}$ "
        f"u točki {array_to_latex(t1_bd(t1_f6_binary_best.element))}"
    )
)
```

Pronađen je minimum vrijednosti $0.000000$ u točki \begin{bmatrix}
  0.000190736 &  0.000190736
\end{bmatrix}

#### Funkcija $f_7$ - binarno

```python
t1_f7_binary_results = simple_ga(
    wellness_function=t1_f7_binary,
    specimen_length=t1_f7_n,
    interval=t1_interval,
    evolutionary_tournament=t1_et_binary,
    max_retries=t1_max_retries,
    max_iterations=t1_max_iterations,
    population_capacity=t1_capacity,
    elitism=t1_elitism,
    epsilon=t1_epsilon,
    mutation_scheduler=t1_mp_schedule,
    encoding_function=t1_be,
    decoding_function=t1_bd,
)
```

    [143] [0. 0.] (-0.02): 100%|██████████| 10000/10000 [00:04<00:00, 2085.72it/s]
    [432] [0. 0.] (-0.02): 100%|██████████| 10000/10000 [00:04<00:00, 2146.38it/s]
    [9679] [-0.  0.] (-0.04): 100%|██████████| 10000/10000 [00:04<00:00, 2089.35it/s]

```python
t1_f7_binary_best = sorted(t1_f7_binary_results, key=lambda x: x.wellness)[-1]

display(
    Markdown(
        f"Pronađen je minimum vrijednosti ${-t1_f7_binary_best.wellness:.06f}$ "
        f"u točki {array_to_latex(t1_bd(t1_f7_binary_best.element))}"
    )
)
```

Pronađen je minimum vrijednosti $0.017346$ u točki \begin{bmatrix}
  0.000190736 &  0.000190736
\end{bmatrix}

---

```python
t1_mp_float_schedule = t1_mp_schedule = lambda x: 0.5 * (np.sin(x/1000) + 1)
```

#### Funkcija $f_1$ - pomična točka

```python
t1_f1_float_results = simple_ga(
    wellness_function=t1_f1_float,
    specimen_length=t1_f1_n,
    interval=t1_interval,
    evolutionary_tournament=t1_et_float,
    max_retries=t1_max_retries,
    max_iterations=t1_max_iterations,
    population_capacity=t1_capacity,
    elitism=t1_elitism,
    epsilon=t1_epsilon,
    mutation_scheduler=t1_mp_schedule,
)
```

    [2831] [-0.83  3.03] (-0.01): 100%|██████████| 10000/10000 [00:03<00:00, 2702.69it/s]
    [3325] [-0.4   3.11] (-0.01): 100%|██████████| 10000/10000 [00:03<00:00, 2783.64it/s]
    [9812] [ 3.13 -0.23] (-0.01): 100%|██████████| 10000/10000 [00:03<00:00, 2728.11it/s]

```python
t1_f1_float_best = sorted(t1_f1_float_results, key=lambda x: x.wellness)[-1]

display(
    Markdown(
        f"Pronađen je minimum vrijednosti ${-t1_f1_float_best.wellness:.06f}$ "
        f"u točki {array_to_latex(t1_f1_float_best.element)}"
    )
)
```

Pronađen je minimum vrijednosti $0.009716$ u točki \begin{bmatrix}
 -0.834055 &  3.02562
\end{bmatrix}

#### Funkcija $f_3$ - pomična točka

```python
t1_f3_float_results = simple_ga(
    wellness_function=t1_f3_float,
    specimen_length=t1_f3_n,
    interval=t1_interval,
    evolutionary_tournament=t1_et_float,
    max_retries=t1_max_retries,
    max_iterations=t1_max_iterations,
    population_capacity=t1_capacity,
    elitism=t1_elitism,
    epsilon=t1_epsilon,
    mutation_scheduler=t1_mp_schedule,
)
```

    [9973] [0.97 2.   3.01 4.01 4.98] (-0.00): 100%|██████████| 10000/10000 [00:03<00:00, 2622.58it/s]
    [6275] [1.   2.   2.98 4.   4.99] (-0.00): 100%|██████████| 10000/10000 [00:03<00:00, 2660.65it/s]
    [6157] [1.03 2.01 2.99 4.01 5.  ] (-0.00): 100%|██████████| 10000/10000 [00:03<00:00, 2710.41it/s]

```python
t1_f3_float_best = sorted(t1_f3_float_results, key=lambda x: x.wellness)[-1]

display(
    Markdown(
        f"Pronađen je minimum vrijednosti ${-t1_f3_float_best.wellness:.06f}$ "
        f"u točki {array_to_latex(t1_f3_float_best.element)}"
    )
)
```

Pronađen je minimum vrijednosti $0.000674$ u točki \begin{bmatrix}
  1.00074 &  1.99733 &  2.97786 &  4.00037 &  4.98675
\end{bmatrix}

#### Funkcija $f_6$ - pomična točka

```python
t1_f6_float_results = simple_ga(
    wellness_function=t1_f6_float,
    specimen_length=t1_f6_n,
    interval=t1_interval,
    evolutionary_tournament=t1_et_float,
    max_retries=t1_max_retries,
    max_iterations=t1_max_iterations,
    population_capacity=t1_capacity,
    elitism=t1_elitism,
    epsilon=t1_epsilon,
    mutation_scheduler=t1_mp_schedule,
)
```

    [9664] [-0.    0.01] (-0.00): 100%|██████████| 10000/10000 [00:03<00:00, 2677.79it/s]
    [9326] [-0.  0.] (-0.00): 100%|██████████| 10000/10000 [00:03<00:00, 2715.31it/s]
    [9531] [-0.2  -3.13] (-0.01): 100%|██████████| 10000/10000 [00:03<00:00, 2739.36it/s]

```python
t1_f6_float_best = sorted(t1_f6_float_results, key=lambda x: x.wellness)[-1]

display(
    Markdown(
        f"Pronađen je minimum vrijednosti ${-t1_f6_float_best.wellness:.06f}$ "
        f"u točki {array_to_latex(t1_f6_float_best.element)}"
    )
)
```

Pronađen je minimum vrijednosti $0.000003$ u točki \begin{bmatrix}
 -0.00139438 &  0.000946987
\end{bmatrix}

#### Funkcija $f_7$ - pomična točka

```python
t1_f7_float_results = simple_ga(
    wellness_function=t1_f7_float,
    specimen_length=t1_f7_n,
    interval=t1_interval,
    evolutionary_tournament=t1_et_float,
    max_retries=t1_max_retries,
    max_iterations=t1_max_iterations,
    population_capacity=t1_capacity,
    elitism=t1_elitism,
    epsilon=t1_epsilon,
    mutation_scheduler=t1_mp_schedule,
)
```

    [9014] [-0.01 -0.01] (-0.13): 100%|██████████| 10000/10000 [00:03<00:00, 2652.64it/s]
    [7061] [-0. -0.] (-0.05): 100%|██████████| 10000/10000 [00:03<00:00, 2725.39it/s]
    [3227] [-0.01 -0.01] (-0.13): 100%|██████████| 10000/10000 [00:03<00:00, 2716.31it/s]

```python
t1_f7_float_best = sorted(t1_f7_float_results, key=lambda x: x.wellness)[-1]

display(
    Markdown(
        f"Pronađen je minimum vrijednosti ${-t1_f7_float_best.wellness:.06f}$ "
        f"u točki {array_to_latex(t1_f7_float_best.element)}"
    )
)
```

Pronađen je minimum vrijednosti $0.050900$ u točki \begin{bmatrix}
 -0.000583279 & -0.00117348
\end{bmatrix}

---

Koje zaključke možete donijeti o uspješnosti GA za pojedinu funkciju? Koje su se funkcije pokazale teškima i zašto?

**Odgovor**:

Vidljivo je da su oba algoritma dobra za različite stvari. Na primjer, algoritam s binarnim zapisom ima problema s $f_3$, no zato iznimno brzo pronalazi minimume $f_6$ i $f_7$. S druge strane, algoritam sa zapisom s pomičnom točkom iznimno brzo nalazi rješenje za $f_3$, a ima problema s $f_6$ i $f_7$.

Moja hipoteza za takvo ponašanje su operatori križanja - operator kod binarnog zapisa koristi **logički i**, koji nije pogodan za $f_3$, ali je za ostale funkcije. S druge strane, u drugom algoritmu koristi se razmjena elemenata između roditelja, čime se vrlo lagano dolazi do rješenja $f_3$, ali ne i ostalih funkcija koje imaju puno lokalnih minimuma.

### Zadatak 2

Provedite GA na funkcijama $f_6$ i $f_7$ mijenjajući dimenzionalnost funkcije $\{1, 3, 6, 10\}$.

```python
t2_length_range = (1, 3, 6, 10)
```

```python
t2_interval = (-50, 150)
```

```python
t2_n_decimals = 3
t2_dim = get_binary_dim_for_interval_and_decimals(
    interval=t2_interval, n_decimals=t2_n_decimals
)
display(
    Markdown(
        f"Za interval $[{t2_interval[0]}, {t2_interval[1]}]$ "
        f"trebat ćemo ${t2_dim}$ binarnih znamenaka da bi "
        f"imali preciznost na ${t2_n_decimals}$ decimale."
    )
)
```

Za interval $[-50, 150]$ trebat ćemo $18$ binarnih znamenaka da bi imali preciznost na $3$ decimale.

```python
t2_be = BinaryEncoder(
    dim=t2_dim, interval=t2_interval
)
t2_bd = BinaryDecoder(
    dim=t2_dim, interval=t2_interval
)
```

```python
t2_f6 = copy.deepcopy(f6)
t2_f7 = copy.deepcopy(f7)
```

```python
t2_f6_binary = Function(lambda x: -t2_f6(t2_bd(x)))
t2_f7_binary = Function(lambda x: -t2_f7(t2_bd(x)))
```

```python
t2_max_retries = 3
t2_max_iterations = 10000
t2_capacity = 20
t2_elitism = 1
t2_epsilon = 1e-6
```

```python
t2_mp_schedule = lambda x: 0.05 * (np.sin(x/1000) + 1)
```

```python
t2_et_binary = EvolutionaryTournament(
    selection=TournamentSelection(),
    crossover=ANDCrossover(),
    mutation=StochasticCorruptionMutation()
)
```

```python
for length in t2_length_range:
    t2_results = simple_ga(
    wellness_function=t2_f6_binary,
    specimen_length=length,
    interval=t2_interval,
    evolutionary_tournament=t2_et_binary,
    max_retries=t2_max_retries,
    max_iterations=t2_max_iterations,
    population_capacity=t2_capacity,
    elitism=t2_elitism,
    epsilon=t2_epsilon,
    mutation_scheduler=t2_mp_schedule,
    encoding_function=t2_be,
    decoding_function=t2_bd,
    )

    _best = sorted(t2_results, key= lambda x: x.wellness)[-1]

    print(f"Za duljinu {length} pronađen je minimum ({-_best.wellness:.06f}).\n\n")
```

    [62] [0.] (-0.00):   1%|          | 62/10000 [00:00<00:06, 1529.65it/s]
    Za duljinu 1 pronađen je minimum (0.000000).

    [855] [0. 0. 0.] (-0.00):   9%|▊         | 855/10000 [00:00<00:04, 1926.03it/s]
    Za duljinu 3 pronađen je minimum (0.000000).

    [3531] [0. 0. 0. 0. 0. 0.] (-0.00):  35%|███▌      | 3531/10000 [00:01<00:03, 1852.74it/s]
    Za duljinu 6 pronađen je minimum (0.000000).

    [7348] [  0.     0.     0.1    0.2    1.56   0.2    0.     0.     0.29 -21.87] (-0.27): 100%|██████████| 10000/10000 [00:05<00:00, 1787.01it/s]
    [6593] [ 25.2  -50.     0.78  25.     0.     3.32   0.2  -35.94   0.   -25.  ] (-0.49): 100%|██████████| 10000/10000 [00:05<00:00, 1795.52it/s]
    [7512] [  0.21 -49.9  -34.37 -25.    25.01 -24.99 -24.61   0.4    3.91 -21.87] (-0.49): 100%|██████████| 10000/10000 [00:05<00:00, 1812.19it/s]
    Za duljinu 10 pronađen je minimum (0.273594).

```python
for length in t2_length_range:
    _best = simple_ga(
    wellness_function=t2_f7_binary,
    specimen_length=length,
    interval=t2_interval,
    evolutionary_tournament=t2_et_binary,
    max_retries=t2_max_retries,
    max_iterations=t2_max_iterations,
    population_capacity=t2_capacity,
    elitism=t2_elitism,
    epsilon=t2_epsilon,
    mutation_scheduler=t2_mp_schedule,
    encoding_function=t2_be,
    decoding_function=t2_bd,
    )

    _best_value = sorted(_best, key= lambda x: x.wellness)[-1]

    print(f"Za duljinu {length} pronađen je minimum ({-_best_value.wellness:.06f}).\n\n")
```

    [14] [0.] (-0.02): 100%|██████████| 10000/10000 [00:04<00:00, 2148.89it/s]
    [54] [0.] (-0.02): 100%|██████████| 10000/10000 [00:04<00:00, 2248.91it/s]
    [83] [0.] (-0.02): 100%|██████████| 10000/10000 [00:04<00:00, 2226.62it/s]
    Za duljinu 1 pronađen je minimum (0.015984).

    [684] [0. 0. 0.] (-0.02): 100%|██████████| 10000/10000 [00:04<00:00, 2069.07it/s]
    [5755] [ 0. -0.  0.] (-0.06): 100%|██████████| 10000/10000 [00:04<00:00, 2024.58it/s]
    [5348] [ 0.    0.   -0.06] (-0.27): 100%|██████████| 10000/10000 [00:04<00:00, 2068.97it/s]
    Za duljinu 3 pronađen je minimum (0.024644).

    [3892] [0. 0. 0. 0. 0. 0.] (-0.03): 100%|██████████| 10000/10000 [00:05<00:00, 1906.14it/s]
    [3357] [0. 0. 0. 0. 0. 0.] (-0.03): 100%|██████████| 10000/10000 [00:05<00:00, 1762.47it/s]
    [9988] [ 0.   -0.37  0.    0.    0.    0.  ] (-0.61): 100%|██████████| 10000/10000 [00:05<00:00, 1762.55it/s]
    Za duljinu 6 pronađen je minimum (0.032666).

    [6210] [  0.     0.     0.78   0.     0.     0.     6.25  12.5    0.01 -50.  ] (-7.51): 100%|██████████| 10000/10000 [00:05<00:00, 1739.90it/s]
    [9236] [  1.61   0.88 -37.5    0.   -18.73   1.57 -25.    12.55 -17.97 -24.99] (-7.70): 100%|██████████| 10000/10000 [00:05<00:00, 1691.23it/s]
    [8363] [-50.     0.42   0.   -25.   -21.87  -9.26   0.03   0.01 -46.87 -43.75] (-9.40): 100%|██████████| 10000/10000 [00:06<00:00, 1626.58it/s]
    Za duljinu 10 pronađen je minimum (7.511748).

Kako povećanje dimenzionalnosti funkcije utječe na ponašanje algoritma?

**Odgovor**:

Što je veća dimenzionalnost, to teže nalazimo rješenje.

### Zadatak 3

Za funkcije $f_6$ i $f_7$ usporedite učinkovitost GA koji koristi binarni prikaz uz preciznost na $4$ decimale (tj. $10^{-4}$) i GA koji koristi prikaz s pomičnom točkom (ostali parametri neka budu jednaki), za dimenzije $3$ i $6$.

```python
t3_interval = (-50, 150)
```

```python
t3_n_decimals = 4
t3_dim = get_binary_dim_for_interval_and_decimals(
    interval=t3_interval, n_decimals=t3_n_decimals
)
display(
    Markdown(
        f"Za interval $[{t3_interval[0]}, {t3_interval[1]}]$ "
        f"trebat ćemo ${t3_dim}$ binarnih znamenaka da bi "
        f"imali preciznost na ${t3_n_decimals}$ decimale."
    )
)
```

Za interval $[-50, 150]$ trebat ćemo $21$ binarnih znamenaka da bi imali preciznost na $4$ decimale.

```python
t3_be = BinaryEncoder(
    dim=t3_dim, interval=t3_interval
)
t3_bd = BinaryDecoder(
    dim=t3_dim, interval=t3_interval
)
```

```python
t3_f6 = copy.deepcopy(f6)
t3_f7 = copy.deepcopy(f7)
```

```python
t3_f6_binary = Function(lambda x: -t3_f6(t3_bd(x)))
t3_f6_float = Function(lambda x: -t3_f6(x))
t3_f7_binary = Function(lambda x: -t3_f7(t3_bd(x)))
t3_f7_float = Function(lambda x: -t3_f7(x))
```

```python
t3_lengths = (3, 6)
```

Rad algoritma ograničite zadanim brojem evaluacija (oko $10^5$).

```python
t3_max_iterations = 100000
```

Inačice algoritma usporedite po uputama u sljedećem odjeljku.

#### Usporedba algoritama i prikaz rezultata

Često je pitanje kako usporediti učinkovitost više algoritama ili istog algoritma za različite vrijednosti nekog parametra. Prije svega potrebno je uočiti da samo jedno izvođenje stohastičkog algoritma nije dovoljno; za svaku inačicu algoritam je potrebno izvesti nekoliko puta (barem $10$ u ovoj vježbi, $30$-$50$ za statistički potkrijepljenu nirvanu) uz *jednaki broj evaluacija* i iz svakog izvođenja zabilježiti dobrotu najbolje jedinke. Tada se nad tim skupom dobrota mogu izračunati različite mjere (srednja vrijednost, medijan, broj 'pogodaka' i slično), koje se koriste za usporedbu. Prilikom ispitivanja u ovoj vježbi iskoristite **broj pogodaka** (vrijednost funkcije cilja ispod $10^{-6}$) kao primarnu te medijan kao sekundarnu mjeru (ako je broj pogodaka jednak). Probajte obrazložiti zašto je medijan dobra mjera (često bolja od srednje vrijednosti).

---

```python
t3_max_retries = 10
t3_capacity = 20
t3_elitism = 1
t3_epsilon = 1e-6
```

```python
t3_pm_schedule = lambda x: 0.05 * (np.sin(x/1000) + 1)
```

```python
t3_folder_name = "data/task-03"
```

```python
t3_et_binary = EvolutionaryTournament(
    selection=TournamentSelection(),
    crossover=ANDCrossover(),
    mutation=StochasticCorruptionMutation()
)
t3_et_float = EvolutionaryTournament(
    selection=TournamentSelection(),
    crossover=ChooseOneCrossover(interval=t1_interval),
    mutation=GaussianMutation()
)
```

```python
t3_result_dict = dict()

for t3_length in t3_lengths:
    for t3_wf_name, t3_wf, t3_et, t3_ef, t3_df in zip(
        ("f6-binary", "f6-float", "f7-binary", "f7-float"),
        (t3_f6_binary, t3_f6_float, t3_f7_binary, t3_f7_float),
        (t3_et_binary, t3_et_float, t3_et_binary, t3_et_float),
        (t3_be, lambda x: x, t3_be, lambda x: x),
        (t3_bd, lambda x: x, t3_bd, lambda x: x)
    ):
        _name = f"{t3_wf_name}_length-{t3_length}"
        _file_name = f"{_name}.pickle"
        _file_path = os.path.join(t3_folder_name, _file_name)

        if not os.path.exists(_file_path):
            t3_results = simple_ga(
                wellness_function=t3_wf,
                specimen_length=t3_length,
                interval=t3_interval,
                evolutionary_tournament=t3_et,
                max_retries=t3_max_retries,
                max_iterations=t3_max_iterations,
                population_capacity=t3_capacity,
                elitism=t3_elitism,
                epsilon=t3_epsilon,
                stop_prematurely=False,
                mutation_scheduler=t3_pm_schedule,
                encoding_function=t3_ef,
                decoding_function=t3_df,
            )

            os.makedirs(t3_folder_name, exist_ok=True)

            with open(_file_path, mode="wb+") as file:
                pickle.dump(t3_results, file)

            print()

        with open(_file_path, mode="rb") as file:
            t3_result_dict[_name] = pickle.load(file)
```

```python
t3_statistic_dict = dict()

for name, specimina in t3_result_dict.items():
    _1 = sum([abs(specimen.wellness) < t3_epsilon for specimen in specimina])
    _2 = np.median(np.array([specimen.wellness for specimen in specimina]), axis=0)

    t3_statistic_dict[name] = (_1, _2)
```

```python
t3_sorted_statistic = sorted(
    list(t3_statistic_dict.items()),
    key=lambda x: (-x[1][0], -x[1][1])
    )

for name, state_name in t3_sorted_statistic:
    print(f"{name}\t{state_name[0]}\t|\t{state_name[1]:.06f}")
```

    f6-binary_length-3	10	|	-0.000000
    f6-binary_length-6	8	|	-0.000000
    f7-binary_length-3	0	|	-0.007209
    f7-binary_length-6	0	|	-0.011827
    f6-float_length-3	0	|	-0.312103
    f6-float_length-6	0	|	-0.490885
    f7-float_length-3	0	|	-3.598449
    f7-float_length-6	0	|	-8.239555

Što možete zaključiti o različitim prikazima rješenja za različite funkcije?

**Odgovor**:

Lako je vidljivo da je najuspješniji binarni zapis, te je $f_7$ naizgleda teža funkcija.

### Zadatak 4

Za funkciju $f_6$ pokušajte pronaći 'idealne' parametre genetskog algoritma. 'Idealne' parametre potrebno je odrediti barem za veličinu populacije (npr. $30$, $50$, $100$, $200$) i vjerojatnost mutacije jedinke (npr. $0.1$, $0.3$, $0.6$, $0.9$) a po želji možete i za još neke druge parametre koje je vaš algoritam koristio.

```python
t4_capacities = (30, 50, 100, 200)
t4_mutation_probabilities = (0.1, 0.3, 0.6, 0.9)
```

Jedan postupak traženja parametara opisan je u nastavku.

#### Kako pronaći dobre parametre

Najčešći postupak optimiranja parametara je jedan po jedan, počevši od parametara veće 'važnosti' (utjecaja na učinkovitost). Primjerice, ako optimiramo veličinu populacije, odabiremo nekoliko vrijednosti za taj parametar i pri tome sve ostale parametre držimo nepromijenjene. Kada smo našli 'idealnu' vrijednost za taj parametar, njega fiksiramo i potom optimiramo idući parametar. Postupak se nastavlja dok to nismo
napravili za sve parametre. Na početku je potrebno definirati početni skup vrijednosti (proizvoljno, po osjećaju). Pronalaženje parametara moguće je i uporabom optimizacijskog algoritma koji kao funkciju cilja dobiva učinkovitost promatranog algoritma na 'nižoj razini' (GA u ovom slučaju), pa je tu riječ o *meta-optimizaciji*. Algoritam na 'višoj razini' može biti deterministički ili stohastički, pa su moguće kombinacije lokalni-GA, ili GA-GA ako se jedan GA koristi za optimizaciju parametara 'nižeg' GA.

Koristite medijan kao mjeru usporedbe i prikažite kretanje učinkovitosti za barem jedan parametar uz pomoć *box-plot prikaza* (opisano u nastavku).

```python
t4_interval = (-50, 150)
```

```python
t4_n_decimals = 6
t4_dim = get_binary_dim_for_interval_and_decimals(
    interval=t4_interval, n_decimals=t4_n_decimals
)
display(
    Markdown(
        f"Za interval $[{t4_interval[0]}, {t4_interval[1]}]$ "
        f"trebat ćemo ${t4_dim}$ binarnih znamenaka da bi "
        f"imali preciznost na ${t4_n_decimals}$ decimale."
    )
)
```

Za interval $[-50, 150]$ trebat ćemo $28$ binarnih znamenaka da bi imali preciznost na $6$ decimale.

```python
t4_be = BinaryEncoder(
    dim=t4_dim, interval=t4_interval
)
t4_bd = BinaryDecoder(
    dim=t4_dim, interval=t4_interval
)
```

```python
t4_f6 = copy.deepcopy(f6)
```

```python
t4_f6_binary = Function(lambda x: -t4_f6(t4_bd(x)))
```

```python
t4_max_iterations = 10000
t4_max_retries = 20
t4_specimen_length = 3
t4_elitism = 1
t4_epsilon = 1e-6
```

```python
t4_folder_name = "data/task-04"
```

```python
t4_et_binary = EvolutionaryTournament(
    selection=TournamentSelection(),
    crossover=ANDCrossover(),
    mutation=StochasticCorruptionMutation()
)
```

```python
t4_result_dict = dict()

for t4_capacity in t4_capacities:
    t4_mutation_probability = t4_mutation_probabilities[0]

    _name = f"capacity-{t4_capacity}_mp-{t4_mutation_probability:.02f}"
    _file_name = f"{_name}.pickle"
    _file_path = os.path.join(t4_folder_name, _file_name)

    if not os.path.exists(_file_path):
        t4_pm_schedule = lambda x: t4_mutation_probability

        t4_results = simple_ga(
            wellness_function=t4_f6_binary,
            specimen_length=t4_specimen_length,
            interval=t4_interval,
            evolutionary_tournament=t4_et_binary,
            max_retries=t4_max_retries,
            max_iterations=t4_max_iterations,
            population_capacity=t4_capacity,
            elitism=t4_elitism,
            epsilon=t4_epsilon,
            stop_prematurely=False,
            mutation_scheduler=t4_pm_schedule,
            encoding_function=t4_be,
            decoding_function=t4_bd,
        )

        os.makedirs(t4_folder_name, exist_ok=True)

        with open(_file_path, mode="wb+") as file:
            _results = np.array([specimen.wellness for specimen in t4_results])

            pickle.dump(
                {
                    "capacity": t4_capacity,
                    "mutation_probability": t4_mutation_probability,
                    "results": _results,
                    "median": float(np.median(_results, axis=0))
                },
                file
            )

        print()

    with open(_file_path, mode="rb") as file:
        t4_result_dict[_name] = pickle.load(file)
```

```python
t4_statistic_dict = dict()

for value in t4_result_dict.values():
    t4_statistic_dict[value["capacity"]] = value["median"]
```

```python
t4_sorted_statistic = sorted(
    list(t4_statistic_dict.items()),
    key=lambda x: -x[1]
    )

t4_best_capacity = t4_sorted_statistic[0][0]

display(
    Markdown(
        f"Optimalna veličina populacije je ${t4_best_capacity}$."
    )
)
```

Optimalna veličina populacije je $50$.

```python
t4_result_dict_2 = dict()

for t4_mutation_probability in t4_mutation_probabilities:
    _name = f"capacity-{t4_capacity}_mp-{t4_mutation_probability:.02f}"
    _file_name = f"{_name}.pickle"
    _file_path = os.path.join(t4_folder_name, _file_name)

    if not os.path.exists(_file_path):
        t4_pm_schedule = lambda x: t4_mutation_probability

        t4_results = simple_ga(
            wellness_function=t4_f6_binary,
            specimen_length=t4_specimen_length,
            interval=t4_interval,
            evolutionary_tournament=t4_et_binary,
            max_retries=t4_max_retries,
            max_iterations=t4_max_iterations,
            population_capacity=t4_best_capacity,
            elitism=t4_elitism,
            epsilon=t4_epsilon,
            stop_prematurely=False,
            mutation_scheduler=t4_pm_schedule,
            encoding_function=t4_be,
            decoding_function=t4_bd,
        )

        os.makedirs(t4_folder_name, exist_ok=True)

        with open(_file_path, mode="wb+") as file:
            _results = np.array([specimen.wellness for specimen in t4_results])

            pickle.dump(
                {
                    "capacity": t4_best_capacity,
                    "mutation_probability": t4_mutation_probability,
                    "results": _results,
                    "median": float(np.median(_results, axis=0))
                },
                file
            )

        print()

    with open(_file_path, mode="rb") as file:
        t4_result_dict_2[_name] = pickle.load(file)
```

```python
t4_statistic_dict_2 = dict()

for value in t4_result_dict_2.values():
    t4_statistic_dict_2[value["mutation_probability"]] = value["median"]
```

```python
t4_sorted_statistic_2 = sorted(
    list(t4_statistic_dict_2.items()),
    key=lambda x: -x[1]
    )

t4_best_mutation_probability = t4_sorted_statistic_2[0][0]

display(
    Markdown(
        f"Optimalna stopa mutacije za veličinu populacije "
        f"${t4_best_capacity}$ je ${t4_best_mutation_probability * 100:.02f}\%$."
    )
)
```

Optimalna stopa mutacije za veličinu populacije $50$ je $10.00\%$.

---

```python
t4_capacity_values = [([-y for y in x["results"]], x["capacity"]) for x in t4_result_dict.values()]
t4_mutation_probability_values = [
    ([-y for y in x["results"]], x["mutation_probability"])
    for x in t4_result_dict_2.values()
]
```

```python
t4_fig, t4_ax = plt.subplots(2, 1, figsize=(16, 12))

t4_ax[0].boxplot([x[0] for x in t4_capacity_values])
t4_ax[0].set_xticklabels([str(x[1]) for x in t4_capacity_values])
t4_ax[0].set_xlabel("Veličina populacije")

t4_ax[1].boxplot([x[0] for x in t4_mutation_probability_values])
t4_ax[1].set_xticklabels([str(x[1]) for x in t4_mutation_probability_values])
t4_ax[1].set_xlabel("Stopa mutacije")

t4_fig.show();
```

### Zadatak 5

Ako ste implementirali turnirsku selekciju, probajte nad nekom težom funkcijom ($f_6$ ili $f_7$) izvesti algoritam koristeći različite veličine turnira. Pomaže li veći turnir algoritmu da pronađe bolja rješenja? Ako ste implementirali selekciju *roulette wheel*,isprobajte više vrijednosti omjera odabira najbolje i najlošije jedinke (skaliranje funkcije cilja) te komentirajte dobivene rezultate (također možete isprobati generacijsku i eliminacijsku varijantu).

```python
t5_tournament_sizes = (3, 5, 7, 8)
```

```python
t5_interval = (-50, 150)
```

```python
t5_n_decimals = 6
t5_dim = get_binary_dim_for_interval_and_decimals(
    interval=t5_interval, n_decimals=t5_n_decimals
)
display(
    Markdown(
        f"Za interval $[{t5_interval[0]}, {t5_interval[1]}]$ "
        f"trebat ćemo ${t5_dim}$ binarnih znamenaka da bi "
        f"imali preciznost na ${t5_n_decimals}$ decimale."
    )
)
```

Za interval $[-50, 150]$ trebat ćemo $28$ binarnih znamenaka da bi imali preciznost na $6$ decimale.

```python
t5_be = BinaryEncoder(
    dim=t5_dim, interval=t5_interval
)
t5_bd = BinaryDecoder(
    dim=t5_dim, interval=t5_interval
)
```

```python
t5_f6 = copy.deepcopy(f6)
```

```python
t5_f6_binary = Function(lambda x: -t5_f6(t5_bd(x)))
```

```python
t5_max_iterations = 10000
t5_max_retries = 20
t5_specimen_length = 3
t5_capacity = 20
t5_elitism = 1
t5_epsilon = 1e-6
```

```python
t5_mp_schedule = lambda x: 0.05 * (np.sin(x/1000) + 1)
```

```python
t5_folder_name = "data/task-05"
```

```python
t5_et_binary = EvolutionaryTournament(
    selection=TournamentSelection(),
    crossover=ANDCrossover(),
    mutation=StochasticCorruptionMutation()
)
```

```python
t5_result_dict = dict()

for t5_tournament_size in t5_tournament_sizes:
    _name = f"tournament-size-{t5_tournament_size}"
    _file_name = f"{_name}.pickle"
    _file_path = os.path.join(t5_folder_name, _file_name)

    if not os.path.exists(_file_path):
        t5_results = simple_ga(
            wellness_function=t5_f6_binary,
            specimen_length=t5_specimen_length,
            interval=t5_interval,
            evolutionary_tournament=t5_et_binary,
            max_retries=t5_max_retries,
            max_iterations=t5_max_iterations,
            population_capacity=t5_capacity,
            elitism=t5_elitism,
            epsilon=t5_epsilon,
            stop_prematurely=False,
            mutation_scheduler=t5_mp_schedule,
            encoding_function=t5_be,
            decoding_function=t5_bd,
        )

        os.makedirs(t5_folder_name, exist_ok=True)

        with open(_file_path, mode="wb+") as file:
            _results = np.array([specimen.wellness for specimen in t5_results])

            pickle.dump(
                {
                    "tournament_size": t5_tournament_size,
                    "results": _results
                },
                file
            )

        print()

    with open(_file_path, mode="rb") as file:
        t5_result_dict[_name] = pickle.load(file)
```

```python
t5_tournament_size_values = [
    ([-y for y in x["results"]], x["tournament_size"])
    for x in t5_result_dict.values()
]
```

```python
t5_fig, t5_ax = plt.subplots(1, 1, figsize=(16, 9))

t5_ax.boxplot([x[0] for x in t5_tournament_size_values])
t5_ax.set_xticklabels([str(x[1]) for x in t5_tournament_size_values])
t5_ax.set_xlabel("Veličina turnira")

t5_fig.show();
```

**Komentar**:

Po boxplotovima se ne vidi neka prevelika pravilnost. Raspršenje u pravilu dominira nad aritmetičkom sredinom, pa odabir veličine turnira s našim postavkama ne mijenja stvari previše. Ovo je indikator da nam je mutacija čak i pri velikom selekcijskom pritisku efektivna u razrješavanju lokalnih minimuma.


# 1.5 决策理论

假设我们的输入向量为 $\mathbf x$，目标向量为 $\mathbf t$，对于回归问题，$\mathbf t$ 是一个连续变量，对于分类问题，$\mathbf t$ 是一个离散变量。联合分布 $p(\mathbf{x, t})$ 反映了这些变量之间的不确定性关系，而使用训练数据求 $p(\mathbf{x, t})$ 作为一种统计推断的过程，通常是十分困难的。

考虑一个医学诊断的例子，假设我们对病人进行了 X 光检查，希望判断病人是否得了癌症。这里，输入 $\mathbf x$ 是 X 光检查的图像，输入是一个标签 $t\in \{0,1\}$，$t=0$ 表示 $\mathcal C_1$类，有癌症，$t=1$ 表示 $\mathcal C_2$类，没有癌症。

我们要计算的分布为 $p(\mathbf{x, t})$ 或者 $p(\mathbf{x}, \mathcal{C}_k)$。

给定一个输入 $\mathbf x$，我们需要判断它属于每一类的概率，即计算后验概率

$$
p(\mathcal C_k|\mathbf x)=\frac{p(\mathbf x|\mathcal C_k) p(\mathcal C_k)}{p(\mathbf x)}
$$

## 1.5.1 最小错误率决策

假设我们的目标是最小化发生错误的概率。

我们需要决定一个准则来决定每个 $\mathbf x$ 所属的类别。这个准则将空间分成了很多个区域 $\mathcal R_k$，每个区域代表一类，表示落在这个区域的点都被标记成了第 $k$ 类。这些区域的边界被叫做决策面（`decision boundaries`）。注意，这些决策区域并不一定需要联系，可能是多个区域的组合成为一类的决策区域。

我们的例子中，错误发生有两种情况：属于 $\mathcal C_1$ 被分为 $\mathcal C_2$ 和属于 $\mathcal C_2$ 被分为 $\mathcal C_1$，因此错误率为

$$
p(mistake) = p(\mathbf x\in \mathcal R_1, \mathcal C_2) + p(\mathbf x\in \mathcal R_2, \mathcal C_1)
= \int_{\mathcal R_1} p(\mathbf x, \mathcal C_2) d\mathbf x + \int_{\mathcal R_2} p(\mathbf x, \mathcal C_1) d\mathbf x
$$

为了最小化错误率，我们需要将 $x$ 标记为概率最大的那个类，即如果 $p(\mathbf x, \mathcal C_1)>p(\mathbf x, \mathcal C_2)$，那么将 $\mathbf x$ 标记为 $\mathcal C_1$

从乘法法则我们知道 $p(\mathbf x, \mathcal C_k)=p(\mathcal C_k|\mathbf x)p(\mathbf x)$，而 $p(\mathbf x)$ 是公共项，因此我们相当于将 $\mathbf x$ 标记为 $p(\mathcal C_k|\mathbf x)$ 最大的类。

对于多类问题，最小化错误率决策可以看出最大化正确率决策，即

$$
p(correct) = \sum_{k=1}^K p(\mathbf x\in \mathcal R_k, \mathcal C_k) = \sum_{k=1}^K \int_{\mathcal R_k} p(\mathbf x, \mathcal C_k) d\mathbf x
$$

下图了说明最小化错误率的原理。

```python
import numpy as np
import scipy as sp
import matplotlib.pyplot as plt
from scipy.stats import norm

%matplotlib inline

xx = np.linspace(-6, 6, 200)
yy_1 = 2 * norm.pdf(xx, loc= -2) + norm.pdf(xx, loc = 2)
yy_2 = 1.5 * norm.pdf(xx, loc = 2)

fig, ax = plt.subplots(figsize=(8, 5))
ax.plot(xx, yy_1, 'k', xx, yy_2, 'k', linewidth=1.5)
ax.set_ylim(-.2, 1.2)
ax.set_xticks([])
ax.set_yticks([])
ax.plot([0.33, 0.33], [0, 1], 'k--',linewidth=1.5)
ax.plot([2, 2], [0, 1], 'k-', linewidth=1.5)
ax.fill_between(xx[xx < 2], yy_2[xx <= 2], color="salmon")
ax.fill_between(xx[xx < 2], np.min(np.vstack([yy_1[xx < 2], yy_2[xx < 2]]), axis=0), color="palegreen")
ax.fill_between(xx[xx > 2], yy_1[xx > 2], color="royalblue")
ax.text(-4, 0.8, r"$p(x, \mathcal{C}_1)$", fontsize="xx-large")
ax.text(2.7, 0.6, r"$p(x, \mathcal{C}_2)$", fontsize="xx-large")
ax.text(0.15, 1.04, r"$x_0$", fontsize="xx-large")
ax.text(1.9, 1.03, r"$\hat{x}$", fontsize="xx-large")
ax.annotate("",
            xy=(-6, -0.07), xycoords='data',
            xytext=(2, -0.07), textcoords='data',
            arrowprops=dict(arrowstyle="<->",
                            connectionstyle="arc3"),
            )
ax.annotate("",
            xy=(2, -0.07), xycoords='data',
            xytext=(6, -0.07), textcoords='data',
            arrowprops=dict(arrowstyle="<->",
                            connectionstyle="arc3"),
            )
ax.text(-2.2, -0.17, '$\mathcal{R}_1$', fontsize="xx-large")
ax.text(3.8, -0.17, '$\mathcal{R}_2$', fontsize="xx-large")

ax.spines['right'].set_color('none')
ax.spines['top'].set_color('none')

ax.spines['bottom'].set_position(('data',0))

plt.show()
```

如图所示，假设决策分界面为 $x=\hat{x}$，在决策面左边的点 $x\geq \hat x$ 分为 $\mathcal C_1$，右边的点为 $\mathcal C_2$ 类。从图中可以看到，彩色的部分都是错误的区域。绿色部分和红色部分表示属于 $\mathcal C_2$ 被错分为 $\mathcal C_1$ ，蓝色部分表示属于 $\mathcal C_1$ 被错分为 $\mathcal C_2$。

如果我们改变 $\hat x$ 的值，红色和绿色的部分的总和始终保持不变，而变化的是红色的部分的面积。从图中可以看出，当决策面 $x=x_0$ 即两条概率曲线的交点时，红色部分的面积为 `0`，此时对应的错误率（彩色面积总和）最小。

## 1.5.2 最小风险决策

在很多情况下，我们的目标要更为复杂。

对于癌症检测的问题，如果一个没有癌症的患者被判断得了癌症，那么结果很可能只是多进行一系列进一步检查以确认；但是如果一个有癌症的患者被判断没有癌症，那么结果很可能是因为没有得到及时治疗而死亡。因此，这两种错误的结果是不一样的，在癌症的样例下，我们更希望第二种错误的样例要更少一些，即使第一种错误会相对增加。

为了量化这种目标，我们引入损失函数（`loss (cost) function`），用来衡量每一种类型的决策所带来的损失。这样我们的目标就变成了最小化损失函数，具体来说，对于一个新来的 $\mathbf x$，其类别为 $\mathcal C_k$，我们的决策将其分为 $\mathcal C_j$ 类（$j$ 可以与 $k$ 相等），我们用 $L_{kj}$ 表示这种决策的损失，例如，在癌症的例子中，我们的 $L$ 矩阵可以被表示为：

| | $\mathcal C_1$ cancer | $\mathcal C_2$ normal |
| --- | --- | --- |
|**$\mathcal C_1$ cancer** | 0 | 1000 |
|**$\mathcal C_2$ normal** | 1 | 0 |

表示如果一个有癌症的被判做没有的损失是 1000，而没有癌症的判做癌症的损失为 1。

为了优化目标函数，我们需要知道给定 $\mathbf x$ 的真实类别，但是这个类别不是确定的，而是由分布 $p(\mathbf x, \mathcal C_k)$ 所决定的，因此，我们转而优化损失的期望：

$$
\mathbb E[L] = \sum_k \sum_j \int_{\mathcal R_j} L_{kj} p(\mathbf x, \mathcal C_k) d\mathbf x
$$

我们的目标是选择决策区域 $\mathcal R_j$ 使得期望最小化。

换句话说，我们对 $\mathbf x$ 决策会使得 $\sum_{k} L_{kj} p(\mathbf x, \mathcal C_k)$ 最小，即 $\sum_{k} L_{kj} p(\mathcal C_k | \mathbf x)$ 最小。

## 1.5.3 拒绝选项

当有多个 $p(\mathcal C_k | \mathbf x)$ 值十分接近时，决策将变得十分困难。

在某些情况下，我们可以选择拒绝做出判断，即拒绝选项（`reject option`）。例如在癌症的例子中，如果机器对决策是否有癌症有困难，那么更好的方法是不做出决策，而让人类专家去处理。

为了达到这个目的，我们可以设定一个阈值 $\theta$，当最大的一个 $p(\mathcal C_k | \mathbf x)$ 值小于 $\theta$ 时，拒绝做出判断。

```python
def sigmoid(x):
    return np.exp(x) / (1 + np.exp(x))

xx = np.linspace(-6, 6, 200)
yy = sigmoid(xx)

fig, ax = plt.subplots(figsize=(8, 5))
ax.plot(xx, yy, 'r', xx, 1 - yy, 'b', linewidth=1.5)
ax.set_xticks([])
ax.set_ylim([-0.15, 1.15])
ax.set_yticks([0, 0.85, 1])
ax.set_yticklabels(["$0.0$", r'$\theta$', "$1.0$"], fontsize="xx-large")

ax.plot([-1.73, -1.73], [0, 0.85], "g")
ax.plot([1.73, 1.73], [0, 0.85], "g")
ax.plot([-6, 6], [0.85, 0.85], "g--")

ax.spines['right'].set_color('none')
ax.spines['top'].set_color('none')

ax.spines['bottom'].set_position(('data',0))

ax.annotate("",
            xy=(1.73, -0.05), xycoords='data',
            xytext=(-1.73, -0.05), textcoords='data',
            arrowprops=dict(arrowstyle="<->",
                            connectionstyle="arc3"))

ax.text(-5.5, 1.05, '$p(\mathcal{C}_1|x)$', fontsize="xx-large")
ax.text(4.5, 1.05, '$p(\mathcal{C}_2|x)$', fontsize="xx-large")
ax.text(-1.5, -0.15, "reject region", fontsize="xx-large")
ax.text(5.5, -0.1, "$x$", fontsize="xx-large")

plt.show()
```

如果设置 $\theta = 1$，那么所有的区域都会被拒绝。如果设置 $\theta < 1/K$，其中 $K$ 是类别数，那么所有的区域都不会被拒绝。

## 1.5.4 推断和决策

我们之前事实上将分类问题分成了两部分：
- 推断部分：使用训练数据建立一个计算 $p(\mathcal{C}_k|\mathbf x)$ 的模型
- 决策部分：使用这些后验概率进行决策

另一种直接的方法是学习一个将输入 $\mathbf x$ 直接映射为决策的函数，这样的函数叫做判别函数（`discriminant function`）。

事实上，我们有三类不同的方法来解决分类问题，按复杂度从高到低排列如下：

1. 建模计算 $p(\mathbf x, \mathcal{C}_k)$，再决策，产生式模型（`generative models`）
- 建模计算 $p(\mathcal{C}_k|\mathbf x)$，再决策，判别式模型（`discriminative models`）
- 直接使用判别函数 $f(\mathbf x)$ 映射到标签类别上，例如二类问题中 $f=0$ 表示 $\mathcal C_1$， $f=1$ 表示 $\mathcal C_2$。

### 比较

方法 1 复杂度最高，通常 $\mathbf x$ 是很高维的数据，因此我们需要很多的数据来保证分布的正确性；$p(\mathcal C_k)$ 通常只是简单的使用训练集的分布决定。其好处在于可以得到 $p(\mathbf x)$，来判断一个新数据点 $\mathbf x$ 在当前模型下的概率，即离群点检测（`outlier detection`）。

如果我们只关注分类决策，那么方法 2 是一个好的选择，因为不需要计算 $p(\mathbf x)$，计算量减少了很多；

方法 3 直接将决策并入了判别函数，比方法 2 更简单。但是我们不再有后验概率的信息。

有后验概率的好处在于：
- 最小风险决策
- 拒绝选项
- 补偿先验概率不均衡的问题
- 结合多个模型

## 1.5.5 回归问题的损失函数

对于回归问题，设输入为 $\mathbf x$，目标值为 $t$，模型输出为 $y(\mathbf x)$，我们的损失函数定义为 $L(t, y(\mathbf x))$，则其期望为：

$$
\mathbb E[L] = \iint L(t, y(\mathbf x)) p(\mathbf x, t)d\mathbf xdt
$$

使用平方误差 $L(t, y(\mathbf x)) = \{t-y(\mathbf x)\}^2$，则损失函数的期望为

$$
\mathbb E[L] = \iint \{t-y(\mathbf x)\}^2 p(\mathbf x, t)d\mathbf xdt
$$

对 $y(\mathbf x)$ 最小化（变分法，参见附录 D）：

$$
\frac{\delta \mathbb E[L]}{\delta y(\mathbf x)}= 2 \int \{y(\mathbf x)-t\} p(\mathbf x, t)dt = 0
$$

我们有

$$
y(\mathbf x) = \frac{\int tp(\mathbf x,t) dt}{p(\mathbf x)}
= \int tp(t|\mathbf x) dt = \mathbb E_t[t|\mathbf x]
$$

即 $y(\mathbf x)$ 是给定 $\mathbf x$ 下 $t$ 的条件期望，又叫回归函数（`regression function`）。

在多元目标函数的情况下，类似的有 $\mathbf{y(x)} = \mathbb E_t[\mathbf{t|x}]$。

```python
xx = np.linspace(-0.9, 0.9, 100)
yy = 4 * xx - np.sin(xx * np.pi)

fig, ax = plt.subplots()
ax.plot(xx, yy, color="red")

ax.set_xlim(-1, 1)
ax.set_ylim(-4, 4)

ax.set_xticks([0])
ax.set_xticklabels([r'$x_0$'], fontsize="xx-large")
ax.set_yticks([0])
ax.set_yticklabels([r'$y(x_0)$'], fontsize="xx-large")

xx = np.linspace(-4, 4, 100)
yy = norm.pdf(xx, scale=0.5) / 5

ax.plot([-1, 0], [0, 0], "g--")
ax.plot([0, 0], [-4, 4], "k")
ax.plot(yy, xx)

ax.text(0.15, -1, r'$p(t|x_0)$', fontsize="xx-large")
ax.text(0.5, 3, r'$y(x)$', fontsize="xx-large")

plt.show()
```

我们也可以使用另一种方式得到这个结果。我们对平方项进行展开：

$$
\begin{align}
\{y(\mathbf x)-t\}^2
& = \{y(\mathbf x)-\mathbb E[t|\mathbf x]+
\mathbb E[t|\mathbf x]-t\}^2 \\
& = \{y(\mathbf x)-\mathbb E[t|\mathbf x]\}^2 + 2\{y(\mathbf x)-\mathbb E[t|\mathbf x]\}\{\mathbb E[t|\mathbf x]-t\} + \{\mathbb E[t|\mathbf x]-t\}^2 \\
\end{align}
$$

带入期望公式，并对 $t$ 进行积分：

$$
\mathbb E[L] =
\int \{y(\mathbf x)-\mathbb E[t|\mathbf x]\}^2 p(\mathbf x) d\mathbf x
+ \int \text{var}[t|\mathbf x]~p(\mathbf x) d\mathbf x
$$

注意到 $y(x)$ 只出现在第一部分，因此，我们为了使其最小，应该有

$$
y(\mathbf x)=\mathbb E[t|\mathbf x]
$$

回归问题通常也有三种方式：

1. 建模 $p(\mathbf x, t)$，然后计算 $p(\mathbf t|x)$，最后得到 $y(\mathbf x)=\mathbb E[t|\mathbf x]$
- 建模 $p(\mathbf t|x)$，得到 $y(\mathbf x)=\mathbb E[t|\mathbf x]$
- 直接对回归函数 $y(x)$ 建模。

有些情况下，平方误差给出的结果可能会很差。除了平方误差，我们还可以使用其他误差函数，例如 Minkowski 误差（平方误差是 $q=2$ 的特殊情况）：

$$
\mathbb E[L_q] = \iint \left|y(\mathbf x)-t\right|^q p(\mathbf x, t)d\mathbf xdt
$$

有如下的结论：

- $q=2$，最小值在条件均值处取到
- $q=1$，最小值在条件中位数处取到
- $q\rightarrow 0$，最小值在条件众数处取到

将 $y(\mathbf x)$ 看作独立变量，使用变分法对 $y(\mathbf x)$ 求导，驻点应该满足：

$$
\int q \left|y(\mathbf x)-t\right|^{q-1} \text{sign}(y(\mathbf x)-t) p(\mathbf x, t) dt = 0
$$

两边同时除以 $p(\mathbf x)$，有

$$
\int_{-\infty}^{y(\mathbf x)} \left|y(\mathbf x)-t\right|^{q-1} p(t|\mathbf x) dt =
\int_{y(\mathbf x)}^{\infty} \left|y(\mathbf x)-t\right|^{q-1} p(t|\mathbf x) dt
$$

当 $q = 1$ 时，我们有

$$
\int_{-\infty}^{y(\mathbf x)} p(t|\mathbf x) dt =
\int_{y(\mathbf x)}^{\infty} p(t|\mathbf x) dt
$$

所以 $y(x)$ 是条件中位数。

当 $q\rightarrow 0$ 时，除了 $t=y(\mathbf x)$ 附近的一小块区域趋近于 0，其他的 $\left|y(\mathbf x)-t\right|^{q} \rightarrow 1$，因此 $\mathbb E[L]$ 趋近于 $1$，减去 $t=y(\mathbf x)$ 附近的一小块区域，因此，对于任意 $\mathbf x$ 值，我们选择 $y(x)$ 取最大的 $p(t|\mathbf x)$ 值能使得减少的值最大。

```python
fig, axes = plt.subplots(2, 2, figsize=[12, 8])
axes = axes.flatten()
qs = [0.3, 1, 2, 10]

xx = np.linspace(-2, 2, 101)

for q, ax in zip(qs, axes):
    ax.plot(xx, np.abs(xx) ** q, 'r')
    ax.set_ylim(0, 2)
    ax.set_xticks([-2, -1, 0, 1, 2])
    ax.set_xticklabels([-2, -1, 0, 1, 2], fontsize="xx-large")
    ax.set_yticks([0, 1, 2])
    ax.set_yticklabels([0, 1, 2], fontsize="xx-large")
    if q in (10, 2):
        ax.set_xlabel("$y-t$", fontsize="xx-large")
    if q in (2, 0.3):
        ax.set_ylabel("$|y-t|^q$", fontsize="xx-large")
    ax.text(-.4, 1.5, "q={}".format(q), fontsize="xx-large")

plt.show()
```