# Quantum Ensemble as Simple Averaging

### Theory and Implementation

##### (Fixed $U_{(i,j)}$ for independent  quantum trajectories)

$$\newcommand{\ket}[1]{\left|{#1}\right\rangle}$$
$$\newcommand{\bra}[1]{\left\langle{#1}\right|}$$
$$\newcommand{\braket}[1]{\left\langle{#1}\right\rangle}$$

### (Step 1) State Preparation

+ Given a 2-qubits $control$ register $(d=2) \implies $  ensemble of $B=2^2$ classifiers

+ $data$ register: *qubit encoding strategy* $\implies$  $N$ $2-$dimensional observations are encoded using $2 \times N$ qubits:
$$
\begin{align}
    \text{data register: } \underset{features}{\big( \overset{4}{\underset{i=1}{\otimes}} \left| x_i \right\rangle \big)}\otimes  \underset{ labels}{\big( \overset{4}{\underset{i=1}{\otimes}} \left| y_i \right\rangle \big)}
\end{align}
$$

where $x_i$ and $y_i$ are encoded as follows:
$$
\begin{align}
    \left| x_i \right\rangle = x_{i,1}\left| 0 \right\rangle +  x_{i,2}\left| 1 \right\rangle
\end{align}
$$

and

if $\left| y_i \right\rangle=\left| 0 \right\rangle$ the $i$-th observation belongs to the class $0$. Otherwise if $\left| y_i \right\rangle=\left| 1 \right\rangle$ the $i$-th observation belongs to the class $1$.

### (Step 1) State Preparation

$$
\begin{align*}
    \left|\Phi_0\right\rangle &=
    \big( H^{\otimes 2} \otimes S_{(x,y)} \big)\left|0\right\rangle \otimes \left|0\right\rangle \otimes \left|0\right\rangle \nonumber \\
        & =
   \left|c_1\right\rangle \otimes \left|c_2\right\rangle \otimes \left|x\right\rangle \left|y\right\rangle \nonumber\\
    & =
    \frac{1}{\sqrt{2}}\big(\left|0\right\rangle+\left|1\right\rangle\big) \otimes \frac{1}{\sqrt{2}}\big(\left|0\right\rangle+\left|1\right\rangle\big) \otimes \left|x_0,x_1,x_2,x_3\right\rangle \left|y_0,y_1,y_2,y_3\right\rangle \end{align*}
$$

where $S_x$ is the routine which encodes in the amplitudes of a qubit a real vector $x$ and $H$ is the Hadamard transformation.

### (Step 2) Sampling in Superposition

The second step regards the generation of $2^d$ different transformations of the training set in superposition, each entangled with a state of the control register. To this end, $d$ steps are necessary, where each step consists in the entanglement of the $i$-th control qubit with two transformations of $\left|x,y\right\rangle$ based on two random unitaries, $U_{(i,1)}$ and $U_{(i,2)}$, for $i = 1,2$.

As shown in the **Appendix A**, *Sampling in Superposition* step leads to the following quantum state:
\begin{align}
\ket{\Phi_{2}}
=  \frac{1}{2}\Big[
 \hspace{.2em} &\ket{00} U_{(2,1)}U_{(1,1)}\ket{x,y}
\nonumber \\ + &
\ket{01} U_{(2,1)}U_{(1,2)}\ket{x,y}
\nonumber \\ + &
\ket{10} U_{(2,2)}U_{(1,1)}\ket{x,y}
\nonumber \\ + &
\ket{11} U_{(2,2)}U_{(1,2)}\ket{x,y}
         \Big]  \nonumber \\
 & \hspace{-2.75em} = \frac{1}{\sqrt{4}} \sum_{b=1}^{4} \ket{b} V_b\ket{x,y}
\end{align}

In order to obtain independend quantum trajectories we provide the following definition for $U_{(i,j)}$:
$$U_{(1,1)} = \text{swap}(x_0,x_2) \times \text{swap}(y_0,y_2)$$
$$U_{(1,2)} = \text{swap}(x_1,x_3) \times \text{swap}(y_1,y_3)$$
$$U_{(2,1)} = \mathbf{I} $$
$$U_{(2,2)} = \text{swap}(x_2,x_3) \times \text{swap}(y_2,y_3)$$

where $ \mathbf{I}$ is the identity matrix. Thus, the step of *Sampling in Superposition* leads to:

\begin{align*}
    \left|\Phi_{2}\right\rangle = \frac{1}{2}\Big[
    & \left|11\right\rangle \left|x_0, x_3, x_1, x_2\right\rangle  \left|y_0, y_3, y_1, y_2\right\rangle
    \\ + &
    \left|10\right\rangle \left|x_2, x_1, x_3, x_0\right\rangle  \left|y_2, y_1, y_3, y_0\right\rangle \nonumber\\
    \hspace{.1em}
    + &
    \left|01\right\rangle \left|x_0, x_3, x_2, x_1\right\rangle \left|y_0, y_3, y_2, y_1\right\rangle \\
    + &
    \left|00\right\rangle \left|x_2, x_1, x_0, x_3\right\rangle \left|y_2, y_1, y_0, y_3\right\rangle
    \Big]
\end{align*}

We can see that swap operations allows to entangle different observations (in terms of the indices of the qubits) to different state of the $control$ register. In particular, if considering the last qubit of the *features* and *labels* (sub-)registers, the above choice for $U_{(i,j)}$ guarantees that each quantum state of the control register is entangled with a different training observation. Using a compact representation:

\begin{align}
     \left|\Phi_{2^{'}}\right\rangle = \frac{1}{2}\Big[
    \left|11\right\rangle \left|x_2\right\rangle  \left|y_2\right\rangle
    +
    \left|10\right\rangle\left|x_0\right\rangle\left|y_0\right\rangle
    +
    \left|01\right\rangle\left|x_1\right\rangle\left|y_1\right\rangle
    +
    \left|00\right\rangle\left|x_3\right\rangle \left|y_3\right\rangle
    \Big] =
    \frac{1}{\sqrt{4}}\sum_{i=0}^{3}\left|i\right\rangle\left|x_i,y_i\right\rangle
\end{align}

Notice that, in this case the $i$-th basis state does not correspond to the integer representation of the binary state.

### (Step 3) Learning via interference

First, the $test$ register is initialised to encode the test set, $\tilde{x}$, considering also an additional register to store the final prediction:

\begin{align}
    (S_{\tilde{x}} \otimes \mathbb{1}) \left|0\right\rangle \left|0\right\rangle =\left|x^{(test)}\right\rangle \left|0\right\rangle
\end{align}

Then, the $data$ and $test$ registers interact via interference using the quantum version of the cosine classifier (gate $F$) to compute the estimates of the target variable:

\begin{align*}
    \left|\Phi_{f}\right\rangle
                = & \Big(\mathbb{1}^{\otimes 2} \otimes F \Big) \left|\Phi_{d}\right\rangle \nonumber \\
                = & (\mathbb{1}^{\otimes d} \otimes F )\Bigg[\frac{1}{\sqrt{2^d}}\sum_{b=1}^{2^d} \left|b\right\rangle \left|x_b, y_b\right\rangle\Bigg] \otimes \left|x^{(test)}\right\rangle \left|0\right\rangle  \nonumber \\
                = & \frac{1}{\sqrt{2^d}}\sum_{b=1}^{2^d} \left|b\right\rangle \left|x_b, y_b\right\rangle\left|x^{(test)}\right\rangle \left|\hat{f}_b\right\rangle
\end{align*}

where $\hat{f}_b$ represents the $b$-th prediction for $\tilde{x}$ given the  $b$-th training set.

### (Step 4) Measurement

\begin{align*}
    \left\langle M \right\rangle = &
    \frac{1}{2^d}\sum_{b=1}^{2^d} \left\langle\hat{f}_b|M|\hat{f}_b\right\rangle =
    \frac{1}{2^d}\sum_{b=1}^{2^d}\left\langle M_b \right\rangle  \nonumber \\
    = & \frac{1}{B} \sum_{b=1}^B \hat{f}_b = \hat{f}_{bag}(\tilde{x}|x,y)
\end{align*}

## Quantum Implementation

```python
# Import pakages and functions
import sys
sys.path.insert(1, '../')

from Utils import *
from modeling import *
```

```python
# load the toy dataset
X_data, Y_data, x_test = load_data_custom()

# Generate the quantum circuit
qc = ensemble_fixed_U(X_data, Y_data, x_test)
```

qc.draw(output='mpl', scale=.6,
        style={'fontsize':15, 'dpi':200})

# Quantum Ensemble as Simple Averaging -  Experiments

This notebook reproduces the results in Section 4.3 of the paper *Quantum Ensemble for Classification* where is shown that the quantum ensemble algorithm is able to compute the expectation value of multiple quantum trajectories in superposition with just one execution of the quantum cosine classifier.

For more details about the theoretical background see **Quantum Ensemble - Independent Trajectories.ipynb**

```python
# Import packages and functions
import sys
sys.path.insert(1, '../')

from Utils import *
from modeling import *

Create the toy dataset reported in Table 1 and execute the (classical) cosine classifiers.
```

```python
# Load data without normalisation
X_data, Y_data, x_test = load_data_custom(normalize = False)

# Create table as shown in the paper (Table 1)
data = pd.DataFrame(X_data, columns = [r'$X_1$', r'$X_2$'])

# Extract the value of the target variable as integer
y = [c[1] for c in Y_data]

# Compute the cosine distance between the training points and the test point
dist = [cosine_similarity([x], [x_test])[0][0] for x in X_data]

# Compute the value of the cosine distance classifier
# for the four training points from the test point
p = [cosine_classifier(x, x_test)[0][0] for x in X_data]

# Extract the probabilities for the test point to be classified in class
# 1 according to the (classical) cosine classifies Equation (16)
probs = []
for i,j in zip(y,p):
    if i == 0:
        probs.append(1-j)
    else:
        probs.append(j)

# Create dataset as in paper (Table 1)
probs = np.array(probs)

# Rename columns
data[r'$y$'] = np.array(y)
data[r'$d($$\cdot$$, $ $x^{(test)})$'] = np.round(dist,2)
data[r'$P($$y^{(test)}$$=1$$|b$ $)$'] = probs

# Rename rows
data.index = [r'$x_1$', r'$x_2$', r'$x_3$', r'$x_4$',]

#Visualize dataset
data
```

```python
# Load normalised data
X_data, Y_data, x_test = load_data_custom()

#Visualisation of quantum cosine classifier
quantum_cosine = quantum_cosine_classifier(X_data[0], x_test, Y_data[0] )
quantum_cosine.draw(output='mpl', scale=.7)
```

For each training point in *data* the quantum cosine classifier is executed to compute the prediction of the target variable for the test point $\tilde{x}$. Thus, given the measurements of the quantum circuts, the target probabilities are retrieved using the function *retrieve_proba*.

```python
qc1 = quantum_cosine_classifier(X_data[0], x_test, Y_data[0] )
r1 = exec_simulator(qc1)
r1 = retrieve_proba(r1)

qc2 = quantum_cosine_classifier(X_data[1], x_test, Y_data[1])
r2 = exec_simulator(qc2)
r2 = retrieve_proba(r2)

qc3 = quantum_cosine_classifier(X_data[2], x_test, Y_data[2])
r3 = exec_simulator(qc3)
r3 = retrieve_proba(r3)

qc4 = quantum_cosine_classifier(X_data[3], x_test, Y_data[3])
r4 = exec_simulator(qc4)
r4 = retrieve_proba(r4)

out = [r1, r2, r3, r4]
```

We compute the average of predictions provided by the four quantum cosine classifiers that corresponds to the classical ensemble prediction, using simple averaging as aggregation strategy.

```python
p0 = [p[0] for p in out]
p1 = [p[1] for p in out]
r_avg = [np.mean(p0), np.mean(p1)]
print(np.mean(p0), np.mean(p1))
```

```python
qc = ensemble_fixed_U(X_data, Y_data, x_test)
qc.draw(output='mpl', scale=.6, #filename='output/ensemble_circuit.png',
        style={'fontsize':15, 'dpi':200})
```

```python
r = exec_simulator(qc, n_shots=8192)
r_ens = retrieve_proba(r)
print(r_ens)
```

```python
# collect the results
output_simulator = [r1, r2, r3, r4, r_avg, r_ens]

data_pred = pd.DataFrame(output_simulator,
                         columns=['p0', 'p1'], index=['qc1','qc2','qc3','qc4','AVG','Ensemble'])
data_pred
# data_pred.to_csv('output/sim_results.csv', index=False)
```

```python
plot_cls(output_simulator, title= '')
```

The probability provided by the quantum cosine classifiers ($f_1$, $f_2$, $f_3$, $f_4$)  are pretty much the same to the classical cosine classifier (*data*). Furthermore, the average of the four classifiers is almost the same to the quantum ensemble prediction where it is required only one execution of the cosine classifier.

### Multiple Experiments

In order to show that the result of quantum ensemble holds regardless of the data reported in *Table 1*, the same experiment is performed on $20$ randomly generated datasets, and the average of the quantum cosine classifiers and the quantum ensemble prediction are compared

```python
seed = 543

n_shots = 8192
N_runs = 20
y_labels =[[0,1], [1,0]]

p1_avg = []
p1_ens = []

np.random.seed(seed)
for run in np.arange(N_runs):
    # print(run)
    x1 = [np.random.randint(1, 9), np.random.randint(1, 9)]
    x2 = [np.random.randint(1, 9), np.random.randint(1, 9)]
    x3 = [np.random.randint(1, 9), np.random.randint(1, 9)]
    x4 = [np.random.randint(1, 9), np.random.randint(1, 9)]

    y1 = y_labels[np.random.randint(0, 2)]
    y2 = y_labels[np.random.randint(0, 2)]
    y3 = y_labels[np.random.randint(0, 2)]
    y4 = y_labels[np.random.randint(0, 2)]

    Y_data = [y1, y2, y3, y4]
    X_data = [x1, x2, x3, x4]

    x_test = [np.random.randint(1, 9), np.random.randint(1, 9)]

    X_data, Y_data, x_test = load_data_custom(X_data, Y_data, x_test = x_test)

    qc1 = quantum_cosine_classifier(X_data[0], x_test, Y_data[0] )
    r1 = exec_simulator(qc1)
    r1 = retrieve_proba(r1)

    qc2 = quantum_cosine_classifier(X_data[1], x_test, Y_data[1])
    r2 = exec_simulator(qc2)
    r2 = retrieve_proba(r2)

    qc3 = quantum_cosine_classifier(X_data[2], x_test, Y_data[2])
    r3 = exec_simulator(qc3)
    r3 = retrieve_proba(r3)

    qc4 = quantum_cosine_classifier(X_data[3], x_test, Y_data[3])
    r4 = exec_simulator(qc4)
    r4 = retrieve_proba(r4)

    out = [r1, r2, r3, r4]

    p0 = [p[0] for p in out]
    p1 = [p[1] for p in out]

    r_avg = [np.mean(p0), np.mean(p1)]
    # print('AVG:', r_avg)

    qc = ensemble_fixed_U(X_data, Y_data, x_test)
    qc = transpile(qc, basis_gates = ['u1', 'u2', 'u3', 'cx'], optimization_level=3)

    r = exec_simulator(qc, n_shots=n_shots)
    r_ens = retrieve_proba(r)
    # print('Ensemble', r_ens)

    out = [r1, r2, r3, r4, r_avg, r_ens]

    p1_avg.append(r_avg[1])
    p1_ens.append(r_ens[1])

avg_vs_ensemble(p1_avg, p1_ens)
```

```python
# Execution on real device
IBMQ.load_account()
provider = IBMQ.get_provider(hub='ibm-q')
provider.backends()
backend_16 = provider.get_backend('ibmq_16_melbourne')
backend_5 = provider.get_backend('ibmq_rome')

def run_real_device(qc, backend, shots=8192):
    job = execute(qc, backend, shots=shots)
    results = job.result()
    r = results.get_counts(qc)
    return r
```

presentation/.ipynb_checkpoints/Gate V - Sampling in Superposition-checkpoint.ipynb


## 高斯判别模型(GDA)
高斯判别模型作分类,假设有两类,则:
\begin{equation}
  y \sim Bernouli(\phi) \\
  x|y=0 \sim N(\mu_0, \Sigma) \\
  x|y=1 \sim N(\mu_1, \Sigma)
\end{equation}
令$\theta = (\phi, \mu_0, \mu_1, \Sigma)$,则参数的似然估计为:
\begin{align*}
L(\theta) &= log(\prod_{i=1}^m P(x^{(i)},y^{(i)})) \\
          &= log(\prod_{i=1}^m P(x^{(i)}|y^{(i)})P(y^{(i)})) \\
          &= \sum_{i=1}^m log(P(x^{(i)}|y^{(i)})) + \sum_{i=1}^m log (P(y^{(i)})) \\
          &= \sum_{i=1}^m (1-y^{(i)})log (P(x^{(i)}|y^{(i)} = 0)) + \sum_{i=1}^m y^{(i)}log(P(x^{(i)}|y^{(i)} = 1)) + \sum_{i=1}^m log(P(y^{(i)})) \\
\end{align*}

求参数的最大似然估计,$argmax_{\theta} L(\theta)$.
参数$\phi$:
\begin{align*}
\frac{\partial L(\theta)}{\partial \phi} &= \frac{\partial \sum_{i=1}^m log(P(y^{(i)}))}{\partial \phi} \\
                                         &= \frac{\partial \sum_{i=1}^m log(\phi^{y^{(i)}}(1-\phi)^{1-y^{(i)}})}{\partial \phi} \\
                                         &= \frac{\partial \sum_{i=1}^m y^{(i)}log(\phi) + (1-y^{(i)})log(1-\phi)}{\partial \phi} \\
                                         &= \sum_{i=1}^m y^{(i)} \frac{1}{\phi} - (1-y^{(i)}) \frac{1}{1-\phi} \\
                                         &= \sum_{i=1}^m I(y^{(i)}=1) \frac{1}{\phi} - I(y^{(i)}=0) \frac{1}{1-\phi} \\
                                         &= a \frac{1}{\phi} + b \frac{1}{1-\phi} \\
                                         &= 0
\end{align*}

$I$为指示函数,$a$为$(y^{(i)}=1)$的样本数量,$b$为$(y^{(i)}=0)$的样本数量,$a+b=m$,$m$为样本总数,令导数为0,则求得:
\begin{equation}
  \phi = \frac{I(y^{(i)}=1)}{m} = \frac{a}{m}
\end{equation}

求参数$\mu_0$:
\begin{align*}
\frac{\partial L(\theta)}{\partial \mu_0} &= \frac{\partial \sum_{i=1}^m (1-y^{(i)})log(P(x^{(i)}|y^{(i)} = 0)) }{\partial \mu_0} \\
                                          &= \frac{\partial \sum_{i=1}^m (1-y^{(i)})(log(\frac{1}{\sqrt{(2\pi)^l|\Sigma|}}) - \frac{1}{2}(x^{(i)} - \mu_0)|\Sigma|^{-1}(x^{(i)} - \mu_0)^T)}{\partial \mu_0} \\
                                          &= \sum_{i=1}^m (1 - y^{(i)})(\Sigma^{-1}(x^{(i)} - \mu_0)) \\
                                          &= \sum_{i=1}^m I(y^{(i)} = 0)(\Sigma^{-1}(x^{(i)} - \mu_0)) \\
                                          &= 0
\end{align*}

令导数为0,从而求得:
\begin{equation}
  \mu_0 = \frac{\sum_{i=1}^m I(y^{(i)}=0) x^{(i)}}{\sum_{i=1}^m I(y^{(i)}=0)}
\end{equation}

同理可求得参数$\mu_1$:
\begin{equation}
  \mu_1 = \frac{\sum_{i=1}^m I(y^{(i)}=1) x^{(i)}}{\sum_{i=1}^m I(y^{(i)}=1)}
\end{equation}

协方差矩阵$\Sigma$,$x$的维度为2,$P(x)$是一个2维高斯分布,
$x-\mu = [x_0,x_1] - [\mu_0,\mu_1]$

$\Sigma = E(x-\mu)^T(x-\mu)$:
\begin{align*}
\Sigma &=  \frac{1}{m} \sum_{i=1}^m (x^{(i)} - \mu^{i})^T(x^{(i)} - \mu^{i})
\end{align*}

```python
import numpy as np
import matplotlib.pyplot as plt

m_samples = 500
t_samples = 200
np.random.seed(0)

# 平移从正态分布中采样的高斯样本,正态分布以0为中心
# 这是一个二维高斯,每一维加上其对应的均值,作为平移偏差
shifted_gaussian = np.random.randn(m_samples, 2) + np.array([6,13])
test_sample_1 = np.random.randn(t_samples, 2) + np.array([3,20])
y_label1 = np.zeros(m_samples)
test_label_1 = np.zeros(t_samples)
# 从正态分布采样数据,乘以协方差矩阵C,用以伸缩正态分布
C1 = np.array([[5, 0], [0.3, 2]])
C2 = np.array([[0.8, -1.68], [3.3, 3.2]])
stretched_gaussian = np.dot(np.random.randn(m_samples * 2, 2), C1)
test_sample_2 = np.dot(np.random.randn(t_samples * 2, 2), C2)
y_label2 = np.ones(m_samples * 2)
test_label_2 = np.ones(t_samples * 2)

X_train = np.vstack([shifted_gaussian, stretched_gaussian])
X_label = np.concatenate([y_label1, y_label2])
X_train_label = np.hstack([X_train, X_label.reshape(X_label.shape[0],1)])
idxs = xrange(X_train_label.shape[0])
idxs = np.random.choice(idxs, m_samples, replace=False)
X_train_label = X_train_label[idxs]

X_test = np.vstack([test_sample_1, test_sample_2])
X_test_label = np.concatenate([test_label_1, test_label_2])

plt.figure(1)
plt.scatter(X_train_label[:,0],X_train_label[:,1])
plt.title('X_train')
plt.figure(2)
plt.scatter(X_test[:,0], X_test[:,1])
plt.title('X_test')
```

```python
def trainGDA(X):
    m,D= X.shape
    D = D-1
    pos = X[X[:,2] == 1][:,:2]
    nag = X[X[:,2] == 0][:,:2]
    a = pos.shape[0]
    b = nag.shape[0]
    phi = float(a) / float(m)

    pos_sum = np.sum(pos, axis=0)
    nag_sum = np.sum(nag, axis=0)
    # [ 5.92345515 13.02799222]
    mu0 = nag_sum / float(b)
    # [0.18769247 0.10985849]
    mu1 = pos_sum / float(a)

    Sigma = np.zeros((D,D))
    for i in range(m):
        x = X[i,:2].reshape(1,D)
        if X[i,2] == 1.0 :
            Sigma += np.dot((x - mu1).T, (x - mu1))
        else:
            Sigma += np.dot((x - mu0).T, (x - mu0))
    Sigma = Sigma / m

    return (phi, mu0, mu1, Sigma)

phi,mu0,mu1,Sigma = trainGDA(X_train_label)
print('phi:',phi)
print('mu0:', mu0)
print('mu1:', mu1)
print('Sigma:', Sigma)

samples_num = 500
samples_num_1 = int(samples_num * phi)
samples_num_2 = int(samples_num * (1-phi))
sample_data_1 = np.random.randn(samples_num_1,2) + mu1
sample_data_1 = sample_data_1.dot(Sigma)
plt.figure(1)
plt.scatter(sample_data_1[:,0],sample_data_1[:,1])
sample_data_2 = np.random.randn(samples_num_2,2) + mu0
sample_data_2 = sample_data_2.dot(Sigma)
plt.figure(2)
plt.scatter(sample_data_2[:,0],sample_data_2[:,1])
sample_data = np.vstack([sample_data_1, sample_data_2])
plt.figure(3)
plt.scatter(sample_data[:,0],sample_data[:,1])
```

```python
def prediction(X,phi,mu0,mu1,Sigma):
    m,D = X.shape

    # 求协方差矩阵的行列式
    sig_det = np.linalg.det(Sigma)

    # 求协方差矩阵的逆
    sig_inv = np.linalg.inv(Sigma)

    ce = X[1].reshape(1,D)
    div_val = 2 * np.pi * np.sqrt(sig_det)
    pre_label = []
    for i in range(m):
        x = X[i].reshape(1,D)
        time_inv_1 = np.dot(x - mu1, sig_inv)
        temp_val_1 = np.dot(time_inv_1, (x - mu1).T)
        time_inv_2 = np.dot(x - mu0, sig_inv)
        temp_val_2 = np.dot(time_inv_2, (x - mu0).T)
        pro_pos = (phi * np.exp(-0.5 * temp_val_1) / div_val).flatten()
        pro_nag = ((1-phi) * np.exp(-0.5 * temp_val_2) / div_val).flatten()
        if pro_pos > pro_nag:
            pre_label.append(1)
        else:
            pre_label.append(0)

    return pre_label

pre_label = prediction(X_test,phi,mu0,mu1,Sigma)
total_num = len(pre_label)
hit_num = X_test_label[X_test_label == pre_label].shape[0]
accuracy = float(hit_num) / float(total_num)
print('accuracy is %d %%' % (accuracy * 100))
```

    accuracy is 97 %


# Model Project - OLG

```python
import numpy as np
from scipy import optimize
import sympy as sm

import matplotlib.pyplot as plt
import ipywidgets as widgets

# autoreload modules when code is run
%load_ext autoreload
%autoreload 2

```

# Model description

In this project we will first consider a an OLG model with a pay as you go system which we will later compare to an OLG model with a fully funded system.

Individuals live for two periods and we assume constant population:

$L_t=L_{t+1}$

The lifetime utility of a household is defined as:

$U_t=ln(c_{1t})+\frac{1}{(1+\rho)}ln(c_{2t+1}), \ \ \rho>-1$

This is subject to the following constraints:

$c_{1t}=w_t(1-\tau)-s_t$

$c_{2t+1}=s_t(1+r_{t+1})+\tau w_{t+1}$

Thus the model has the following parameters:

$L_t$: The population at time $t$

$U_t$: Lifetime household utility

$\rho$: Consumption discount rate

$c_{1t}$: Consumption of the young

$c_{2t+1}$: Consumption of the old

$w_t$: Wage

$r_t$: Return on capital

$s_t$: Savings

```python
# Define symbols for sympy
K_t = sm.symbols('K_t')
L_t = sm.symbols('L_t')
A = sm.symbols('A')
alpha = sm.symbols('alpha')
delta = sm.symbols('delta')
rho = sm.symbols('rho')
w_t = sm.symbols('w_t')
r_t = sm.symbols('r_t')
k_t = sm.symbols('k_t')
k_plus = sm.symbols('k_t+1')
Pi = sm.symbols('Pi_t')

s_t = sm.symbols('s_t')
U_t = sm.symbols('U_t')
U = sm.symbols('U_t')
r_plus = sm.symbols('r_t+1')
k = sm.symbols('k^*')

d_t = sm.symbols('d_t')
d_plus = sm.symbols('d_t+1')
tau = sm.symbols('tau')
s_payg = sm.symbols('s_payg')
w_plus = sm.symbols('w_t+1')
k_payg = sm.symbols('k_payg')
U_payg = sm.symbols('U_paygt')
kpaygss = sm.symbols('k_payg^*')
```

```python
#In order to derive the wage (w) and return on capital (r) we need to solve the firms profit maximization problem.

#Define the profit function
profit = A*(K_t**alpha)*(L_t**(1-alpha))-w_t*L_t-(r_t+delta)*K_t

#Setup the FOC's
foc1=sm.diff(profit,K_t)
foc2=sm.diff(profit,L_t)

#Solve FOC's
solve_foc1=sm.solve(foc1,r_t)[0]
solve_foc2=sm.solve(foc2,w_t)[0]

#Print eqations
print('The profit of a given firm is defined as:')
display(sm.Eq(Pi,profit))
print('By taking the first order conditions to this, we get the expressions for w an r:')
display(sm.Eq(r_t,solve_foc1))
display(sm.Eq(w_t,solve_foc2))
```

    The profit of a given firm is defined as:

$\displaystyle \Pi_{t} = A K_{t}^{\alpha} L_{t}^{1 - \alpha} - K_{t} \left(\delta + r_{t}\right) - L_{t} w_{t}$

    By taking the first order conditions to this, we get the expressions for w an r:

$\displaystyle r_{t} = \frac{A K_{t}^{\alpha} L_{t} L_{t}^{- \alpha} \alpha}{K_{t}} - \delta$

$\displaystyle w_{t} = A K_{t}^{\alpha} L_{t}^{- \alpha} \left(1 - \alpha\right)$

We now assume full capital depreciation $\delta=1$

```python
#Setup equations
sample_r=alpha*A*k_t**(alpha-1)-1
sample_w=(1-alpha)*A*k_t**alpha

#Display them
print('By using the marginal products of capital and labour we can simplify these expressions to:')
display(sm.Eq(r_t,sample_r))
display(sm.Eq(w_t,sample_w))
```

    By using the marginal products of capital and labour we can simplify these expressions to:

$\displaystyle r_{t} = A \alpha k_{t}^{\alpha - 1} - 1$

$\displaystyle w_{t} = A k_{t}^{\alpha} \left(1 - \alpha\right)$

We now move unto the household optimization problem and determine the savings rate.

```python
#We solve the household optimization problem and determine savings (s)

#Substitute constraints into utility function
U_t = sm.log(w_t*(1-tau)-s_t)+(1/(1+rho))*sm.log(s_t*(1+r_plus)+tau*w_plus)

#Take the FOC in regards to savings
FOC_U = sm.diff(U_t,s_t)

#Isolate savings
isolate_s = sm.solve(FOC_U,s_t)[0]

#Print results
print('Substitute the constraints into the utility function')
display(sm.Eq(U,U_t))
print('Take the FOC in regards to savings:')
display(sm.Eq(0,FOC_U))
print('Isolate savings:')
display(sm.Eq(s_t,isolate_s))
print('we can simplify this to:')
display(sm.Eq(s_t,-tau*w_plus*(rho+1)/((r_plus+1)*(rho+2))+w_t*(1-tau)/(rho+2)))

```

    Substitute the constraints into the utility function

$\displaystyle U_{t} = \log{\left(- s_{t} + w_{t} \left(1 - \tau\right) \right)} + \frac{\log{\left(s_{t} \left(r_{t+1} + 1\right) + \tau w_{t+1} \right)}}{\rho + 1}$

    Take the FOC in regards to savings:

$\displaystyle 0 = \frac{r_{t+1} + 1}{\left(\rho + 1\right) \left(s_{t} \left(r_{t+1} + 1\right) + \tau w_{t+1}\right)} - \frac{1}{- s_{t} + w_{t} \left(1 - \tau\right)}$

    Isolate savings:

$\displaystyle s_{t} = \frac{- r_{t+1} \tau w_{t} + r_{t+1} w_{t} - \rho \tau w_{t+1} - \tau w_{t} - \tau w_{t+1} + w_{t}}{r_{t+1} \rho + 2 r_{t+1} + \rho + 2}$

    we can simplify this to:

$\displaystyle s_{t} = - \frac{\tau w_{t+1} \left(\rho + 1\right)}{\left(r_{t+1} + 1\right) \left(\rho + 2\right)} + \frac{w_{t} \left(1 - \tau\right)}{\rho + 2}$

# Steady state

We now wish to determine the steady state level of capital $k^*$

```python
# We define capital accumulation using the previuos expressions and solve for the steady state level of capital.

#capital accumulation
print('capital accumulation is given by')
display(sm.Eq(k_plus,-tau*w_plus*(rho+1)/((r_plus+1)*(rho+2))+w_t*(1-tau)/(rho+2)))
print('We insert the expressions for w and r and get')
display(sm.Eq(k_plus,(A*k_t**(alpha)*(1-alpha)*(1-tau))/(rho+2)-(k_plus**(1-alpha)*tau*w_plus*(rho+1))/(A*alpha*(rho+2))))
```

    capital accumulation is given by

$\displaystyle k_{t+1} = - \frac{\tau w_{t+1} \left(\rho + 1\right)}{\left(r_{t+1} + 1\right) \left(\rho + 2\right)} + \frac{w_{t} \left(1 - \tau\right)}{\rho + 2}$

    We insert the expressions for w and r and get

$\displaystyle k_{t+1} = \frac{A k_{t}^{\alpha} \left(1 - \alpha\right) \left(1 - \tau\right)}{\rho + 2} - \frac{k_{t+1}^{1 - \alpha} \tau w_{t+1} \left(\rho + 1\right)}{A \alpha \left(\rho + 2\right)}$

```python
capital_accumulation = 1/(1+(1+rho)/(2+rho)*(1-alpha)/alpha*tau)*(((1-alpha)*(1-tau))/(2+rho)*A*k_t**alpha)
capital_accumulation
```

$\displaystyle \frac{A k_{t}^{\alpha} \left(1 - \alpha\right) \left(1 - \tau\right)}{\left(1 + \frac{\tau \left(1 - \alpha\right) \left(\rho + 1\right)}{\alpha \left(\rho + 2\right)}\right) \left(\rho + 2\right)}$

In the steady state it is true that

$k_{t+1}=k_t=k^*$

We can therefore rewrite the above to:

```python
#Define the steady state capital
ssk=sm.Eq(k,(1/(1+(1+rho)/(2+rho)*(1-alpha)/(alpha)*tau)*((1-alpha)*(1-tau))/(2+rho)*A)**(1/(1-alpha)))
display(ssk)

```

$\displaystyle k^{*} = \left(\frac{A \left(1 - \alpha\right) \left(1 - \tau\right)}{\left(1 + \frac{\tau \left(1 - \alpha\right) \left(\rho + 1\right)}{\alpha \left(\rho + 2\right)}\right) \left(\rho + 2\right)}\right)^{\frac{1}{1 - \alpha}}$

## Analytical solution

```python
#Create a function to determine capital level in steady state given a set of parameters
ssk_solve = sm.solve(ssk,k)
ss_func = sm.lambdify((alpha,tau,rho,A),ssk_solve)

# Assign arbitrary values to the parameters
value_alpha = 0.20
value_tau = 0.25
value_rho = 0.1
value_A = 2

#Run the function with the values
res=ss_func(value_alpha,value_tau,value_rho,value_A)

#Display results
print('Thus we find that the ss capital level, given the parameters, is: '+str(k)+' = '+str(res[0]))
```

    Thus we find that the ss capital level, given the parameters, is: k^* = 0.29345335876374135

We can also check this by doing it manually.

```python
#Define function
def ss_alt(alpha, tau, rho, A):
    #Basically the equation for the steady state capital
    return((1/(1+(1+rho)/(2+rho)*(1-alpha)/(alpha)*tau)*((1-alpha)*(1-tau))/(2+rho)*A)**(1/(1-alpha)))
#Print results with given parameters
print(f'The steady state capital in the pay as you go model is: '+str(ss_alt(value_alpha,value_tau,value_rho,value_A)))

```

    The steady state capital in the pay as you go model is: 0.29345335876374135

# Numerical solution

```python
#Define a lambda function for the capital accumulation function and optimize in regards to k
obj_kpayg = lambda kpayg: kpayg - (1/(1+(1+value_rho)/(2+value_rho)*(1-value_alpha)/value_alpha*value_tau)*(((1-value_alpha)*(1-value_tau))/(2+value_rho)*value_A*kpayg**value_alpha))

r_payg = optimize.root_scalar(obj_kpayg, bracket=[0.1, 100], method='brentq')

#Print results
print(r_payg)
print(f'k^*={r_payg.root:.4f}')
```

          converged: True
               flag: 'converged'
     function_calls: 8
         iterations: 7
               root: 0.2934533587636173
    k^*=0.2935

# OLG Fully Funded system

Next an OLG model with a fully funded system is solved.

# Model description

The lifetime utility of a household is the same as in the pay as you go model:

$U_t=ln(c_{1t})+\frac{1}{(1+\rho)}ln(c_{2t+1}), \ \ \rho>-1$

The constraint now looks as following, the government takes part of their wage and invest it. In period 2 they receive this amount plus the interest earned, plus any additional savings they have made:

$c_{1t}=w_t(1-\tau)-s_t$

$c_{2t+1}=(s_t+w_t\tau)(1+r_{t+1})$

There is still full depreciation and a constant population. The firms maximisation problem does not change.

The approach for solving this model is the same as for the pay as you go model.

```python
#We solve the household optimization problem and determine savings (s)

#Substitute constraints into utility function
U_t2 = sm.log(w_t*(1-tau)-s_t)+(1/(1+rho))*sm.log((s_t+w_t*tau)*(1+r_plus))

#Take the FOC in regards to savings
FOC_U2 = sm.diff(U_t2,s_t)

#Isolate savings
isolate_s2 = sm.solve(FOC_U2,s_t)[0]

#Print results
print('Substitute the constraints into the utility function')
display(sm.Eq(U,U_t2))
print('Take the FOC in regards to savings:')
display(sm.Eq(0,FOC_U2))
print('Isolate savings:')
display(sm.Eq(s_t,isolate_s2))

#Reducing
print('Reduce this to:')
s2 = w_t*(1/(2+rho)-tau)
s2 = sm.Eq(s_t,s2)
display(s2)

print('Substitute in wage:')
display(s2.subs(w_t, sample_w))

#Substitute in k_t
cap=s2.subs(w_t, sample_w)

#Display the cap accum
print('From this we can get the capital accumulation expression')
display(cap.subs(s_t, k_plus))
```

    Substitute the constraints into the utility function

$\displaystyle U_{t} = \log{\left(- s_{t} + w_{t} \left(1 - \tau\right) \right)} + \frac{\log{\left(\left(r_{t+1} + 1\right) \left(s_{t} + \tau w_{t}\right) \right)}}{\rho + 1}$

    Take the FOC in regards to savings:

$\displaystyle 0 = - \frac{1}{- s_{t} + w_{t} \left(1 - \tau\right)} + \frac{1}{\left(\rho + 1\right) \left(s_{t} + \tau w_{t}\right)}$

    Isolate savings:

$\displaystyle s_{t} = \frac{w_{t} \left(- \rho \tau - 2 \tau + 1\right)}{\rho + 2}$

    Reduce this to:

$\displaystyle s_{t} = w_{t} \left(- \tau + \frac{1}{\rho + 2}\right)$

    Substitute in wage:

$\displaystyle s_{t} = A k_{t}^{\alpha} \left(1 - \alpha\right) \left(- \tau + \frac{1}{\rho + 2}\right)$

    From this we can get the capital accumulation expression

$\displaystyle k_{t+1} = A k_{t}^{\alpha} \left(1 - \alpha\right) \left(- \tau + \frac{1}{\rho + 2}\right)$

# Steady State

In the steady state it is true that

$k_{t+1}=k_t=k^*$

We can therefore rewrite the above to:

```python
#Define the steady state capital expression
ssk2=sm.Eq(k,((A*(alpha*rho*tau+2*alpha*tau-alpha-rho*tau-2*tau+1))/(rho+2))**(-1/(alpha-1)))
#display it
display(ssk2)

```

$\displaystyle k^{*} = \left(\frac{A \left(\alpha \rho \tau + 2 \alpha \tau - \alpha - \rho \tau - 2 \tau + 1\right)}{\rho + 2}\right)^{- \frac{1}{\alpha - 1}}$

# Analytical Solution

```python
#Create a function to determine capital level in steady state given a set of parameters
ssk_solve2 = sm.solve(ssk2,k)
ss_func2 = sm.lambdify((alpha,tau,rho,A),ssk_solve2)

#Run the function with the values
res2=ss_func2(value_alpha,value_tau,value_rho,value_A)

#Display results
print('Thus we find that the ss capital level, given the parameters, is: '+str(k)+' = '+str(res2[0]))

```

    Thus we find that the ss capital level, given the parameters, is: k^* = 0.28070029686156567

# Nummerical Solution

We now run the nummerical analysis

```python
display(cap.subs(s_t, k_plus))
display((A*k_t**alpha*(1-alpha))*(-tau+1/(rho+2)))
```

$\displaystyle k_{t+1} = A k_{t}^{\alpha} \left(1 - \alpha\right) \left(- \tau + \frac{1}{\rho + 2}\right)$

$\displaystyle A k_{t}^{\alpha} \left(1 - \alpha\right) \left(- \tau + \frac{1}{\rho + 2}\right)$

```python
#Define a lambda function for the capital accumulation function and optimize in regards to k
obj_kff = lambda kff: kff - ((value_A*kff**value_alpha*(1-value_alpha))*(-value_tau+1/(value_rho+2)))

r_ff = optimize.root_scalar(obj_kff, bracket=[0.1, 100], method='brentq')

#Print results
print(r_ff)
print(f'k^*={r_ff.root:.4f}')
```

          converged: True
               flag: 'converged'
     function_calls: 8
         iterations: 7
               root: 0.2807002968610902
    k^*=0.2807

# Comparision of the fully funded system and the payg system

From this we see that in the payg system there was a capital level of 0.293 and in the fully funded it was 0.281. It would therefore seem that with the given parameters that the capital level is higher in the payg-model.

One could furthermore try to change the parameters, like the tax level to observe the changes which we will do now.

```python
# Assign a new tax value (tau)
value_tau = 0.35

#Numerical for payg with new tax level
obj_kpayg = lambda kpayg: kpayg - (1/(1+(1+value_rho)/(2+value_rho)*(1-value_alpha)/value_alpha*value_tau)*(((1-value_alpha)*(1-value_tau))/(2+value_rho)*value_A*kpayg**value_alpha))

r_payg = optimize.root_scalar(obj_kpayg, bracket=[0.1, 100], method='brentq')

#Print results
print('For a tax value of 35% we observe in the payg-system: ')
print(f'k^*={r_payg.root:.4f}')

#numerical for ff with new tax level
obj_kff = lambda kff: kff - ((value_A*kff**value_alpha*(1-value_alpha))*(-value_tau+1/(value_rho+2)))

r_ff = optimize.root_scalar(obj_kff, bracket=[0.1, 100], method='brentq')

#Print results
print('And in the fully funded system we observe: ')
print(f'k^*={r_ff.root:.4f}')
```

    For a tax value of 35% we observe in the payg-system:
    k^*=0.2089
    And in the fully funded system we observe:
    k^*=0.1353

```python
# Assign a new tax value (tau)
value_tau = 0.05

#Numerical for payg with new tax level
obj_kpayg = lambda kpayg: kpayg - (1/(1+(1+value_rho)/(2+value_rho)*(1-value_alpha)/value_alpha*value_tau)*(((1-value_alpha)*(1-value_tau))/(2+value_rho)*value_A*kpayg**value_alpha))

r_payg = optimize.root_scalar(obj_kpayg, bracket=[0.1, 100], method='brentq')

#Print results
print('For a tax value of 5% we observe in the payg-system: ')
print(f'k^*={r_payg.root:.4f}')

#numerical for ff with new tax level
obj_kff = lambda kff: kff - ((value_A*kff**value_alpha*(1-value_alpha))*(-value_tau+1/(value_rho+2)))

r_ff = optimize.root_scalar(obj_kff, bracket=[0.1, 100], method='brentq')

#Print results
print('And in the fully funded system we observe: ')
print(f'k^*={r_ff.root:.4f}')
```

    For a tax value of 5% we observe in the payg-system:
    k^*=0.5894
    And in the fully funded system we observe:
    k^*=0.6197

```python
tau_vec = np.linspace(0,0.4,100)
obj_kpayg = lambda kpayg, value_tau: kpayg - (1/(1+(1+value_rho)/(2+value_rho)*(1-value_alpha)/value_alpha*value_tau)*(((1-value_alpha)*(1-value_tau))/(2+value_rho)*value_A*kpayg**value_alpha))

r_payg_vec = [optimize.root_scalar(obj_kpayg, bracket=[0.0001, 100], method='brentq',args=(tau_val)).root for tau_val in tau_vec ]

obj_kff = lambda kff, value_tau: kff - ((value_A*kff**value_alpha*(1-value_alpha))*(-value_tau+1/(value_rho+2)))

r_ff_vec = [optimize.root_scalar(obj_kff, bracket=[0.0001, 100], method='brentq',args=(tau_val)).root for tau_val in tau_vec]
```

```python
plt.plot(tau_vec,r_payg_vec,color='red')
plt.plot(tau_vec,r_ff_vec);
```

# Conclusion

From these results it would seem that capital levels in the fully funded system are actually higher given that the tax is low.
It would therefore seem that by increasing taxes the capital levels in both systems fall, but that the fully funded system falls at a greater rate than the payg-system.

In conclusion taxes seems to affect the fully funded system more severely than the payg-system.

```python

```


# Модель расчетов

Расчет профиля и навески разбит на несколько этапов

1. Оптимизация профиля по ионному балансу.
   Профиль оптимизируется по сере и хлору
2. Перерасчет профиля по требуемому EC
3. Перерасчет профиля по коэфициентам
4. Расчет навески удобрений

# Расчет EC

## Формула Зоневельда

Выполняется при условии того что ионный баланс равен нулю

$$ \left(
\frac{m_{NH_4}}{M_N} +
     \frac{2  m_{Ca}}{M_{Ca}} +
     \frac{2  m_{Mg}}{M_{Mg}} +
     \frac{m_{K}}{M_{K}}
\right)
     -
     \left(
     \frac{m_{NO_3}}{M_N} +
     \frac{m_{P}}{M_{P}} +
     \frac{2m_{S}}{M_{S}} +
     \frac{m_{Cl}}{M_{Cl}}
     \right)
     = 0 $$

Тогда:

$$ EC = 0.095
    \left(
     \frac{m_{NH_4}}{M_N} +
     \frac{2  m_{Ca}}{M_{Ca}} +
     \frac{2  m_{Mg}}{M_{Mg}} +
     \frac{m_{K}}{M_{K}}
     \right) + 0.19 $$

Где: \
$M$ --- молярная масса\
$m$ --- профиль, в мг/л (ppm)

```python
m_NO3 = 220
m_NH4 = 20
m_P = 40
m_K = 280
m_Ca = 140
m_Mg = 56
m_S = 51.07
m_Cl = 0

M_N = 14.0067
M_Ca = 40.078
M_Mg = 24.305
M_K = 39.0983
M_P = 30.973
M_S = 32.065
M_Cl = 35.45

cations = ((m_NH4/ M_N) + ((2*m_Ca) / M_Ca) + ((2*m_Mg) / M_Mg) + (m_K / M_K) )

anions = ((m_NO3/ M_N) + (m_P / M_P) + ((2*m_S)/ M_S) + (m_Cl/ M_Cl))

EC = (0.095 * cations) + 0.19
print(cations, anions)
print(f'I = {cations-anions:0.2f}')
print(f'EC = {EC:.3f}')

```

    20.183806662456348 20.183620939926474
    I = 0.00
    EC = 2.107

## Ионный баланс профиля

Оптимизация профиля достижением нулевого ионного баланса (сумма катионов и анионов должна быть равна нулю)

Основой является формула ионного баланса

$$ \left(
\frac{m_{NH_4}}{M_N} +
     \frac{2  m_{Ca}}{M_{Ca}} +
     \frac{2  m_{Mg}}{M_{Mg}} +
     \frac{m_{K}}{M_{K}}
\right)
     -
     \left(
     \frac{m_{NO_3}}{M_N} +
     \frac{m_{P}}{M_{P}} +
     \frac{2m_{S}}{M_{S}} +
     \frac{m_{Cl}}{M_{Cl}}
     \right)
     = 0 $$

Баланс производится по сере или кальцию, в зависимости от того что крутим

```python
# Install a pip package in the current Jupyter kernel
import sys
!{sys.executable} -m pip install sympy numpy scipy

import sympy as sym
from sympy import Eq
sym.init_printing()
from IPython.display import Math, display

```

    Requirement already satisfied: sympy in /home/apkawa/.local/lib/python3.9/site-packages (1.7.1)
    Requirement already satisfied: numpy in /usr/lib/python3/dist-packages (1.19.4)
    Collecting scipy
      Downloading scipy-1.6.0-cp39-cp39-manylinux1_x86_64.whl (27.3 MB)
         |████████████████████████████████| 27.3 MB 12.2 MB/s eta 0:00:01
    Requirement already satisfied: mpmath>=0.19 in /home/apkawa/.local/lib/python3.9/site-packages (from sympy) (1.1.0)
    Installing collected packages: scipy
    Successfully installed scipy-1.6.0

```python
import sympy as sym
sym.init_printing()
elements = ['NO3', 'NH4', 'P', 'K', 'Ca', 'Mg', 'S', 'Cl']

m = dict(zip(elements, sym.symbols([f'm_{e}' for e in elements])))
M = dict(zip(elements, sym.symbols([f'M_{e}' for e in elements])))

repl = {
m['NO3'] : 220,
m['NH4'] : 20,
m['P'] : 40,
m['K'] : 280,
m['Ca'] : 140,
m['Mg'] : 56,
m['S']: 51.07,
m['Cl']: 0,
    # Молярные массы
M['NO3'] : 14.0067, M['NH4'] : 14.0067, M['Ca'] : 40.078,
M['Mg'] : 24.305, M['K'] : 39.0983, M['P'] : 30.973, M['S'] : 32.065, M['Cl']: 35.45,
}

# Уравнение ионного баланса
Cations = ((m['NH4']/ M['NH4']) + ((2*m['Ca']) / M['Ca']) + ((2*m['Mg']) / M['Mg']) + (m['K'] / M['K']))
Anions = ((m['NO3']/ M['NO3']) + (m['P'] / M['P']) + ((2*m['S'])/ M['S']) + (m['Cl']/ M['Cl']))
Ionic_balance = sym.Eq((Cations - Anions), 0)

I = (Cations - Anions).subs(repl)
print("Проверка что профиль сбалансирован")
print(f"I = {I:0.3f}")

# Решаем уравнение через серу
solved_S = sym.solve([Ionic_balance],(m['S']))[m['S']]

mS = solved_S.subs(repl)

print(f'm_S = {mS:.2f}')

I = (Cations - Anions).subs({**repl, **{m['S']: mS}})
print(f"I = {I:0.3f}")

print("Сера через кальций")

display(sym.Eq(m['S'], solved_S))
r2 = {**repl, **{
    m['NO3'] : 200,
    m['NH4'] : 20,
    m['P'] : 40,
    m['K'] : 180,
    m['Ca'] : 200,
    m['Mg'] : 50,
}}
mS = solved_S.subs(r2)
print(f'm_S = {mS:.2f}')
r2[m['S']] = mS
I = (Cations - Anions).subs(r2)
print(f"I = {I:0.3f}")

print("Кальций через серу")
r2 = {**r2, **{m['S']: mS}}

solved_Ca = sym.solve([Ionic_balance],(m['Ca']))[m['Ca']]
display(sym.Eq(m['Ca'], solved_Ca))
mCa = solved_Ca.subs(r2)
print(f'm_Ca = {mCa:.2f}')

I = (Cations - Anions).subs({**r2, **{m['Ca']: mCa}})
print(f"I = {I:0.3f}")

```

    Проверка что профиль сбалансирован
    I = 0.000
    m_S = 51.07
    I = 0.000
    Сера через кальций

$\displaystyle m_{S} = \frac{- M_{Ca} M_{Cl} M_{K} M_{Mg} M_{NH4} M_{NO3} M_{S} m_{P} - M_{Ca} M_{Cl} M_{K} M_{Mg} M_{NH4} M_{P} M_{S} m_{NO3} + M_{Ca} M_{Cl} M_{K} M_{Mg} M_{NO3} M_{P} M_{S} m_{NH4} + 2 M_{Ca} M_{Cl} M_{K} M_{NH4} M_{NO3} M_{P} M_{S} m_{Mg} + M_{Ca} M_{Cl} M_{Mg} M_{NH4} M_{NO3} M_{P} M_{S} m_{K} - M_{Ca} M_{K} M_{Mg} M_{NH4} M_{NO3} M_{P} M_{S} m_{Cl} + 2 M_{Cl} M_{K} M_{Mg} M_{NH4} M_{NO3} M_{P} M_{S} m_{Ca}}{2 M_{Ca} M_{Cl} M_{K} M_{Mg} M_{NH4} M_{NO3} M_{P}}$

    m_S = 73.05
    I = 0.000
    Кальций через серу

$\displaystyle m_{Ca} = \frac{2 M_{Ca} M_{Cl} M_{K} M_{Mg} M_{NH4} M_{NO3} M_{P} m_{S} + M_{Ca} M_{Cl} M_{K} M_{Mg} M_{NH4} M_{NO3} M_{S} m_{P} + M_{Ca} M_{Cl} M_{K} M_{Mg} M_{NH4} M_{P} M_{S} m_{NO3} - M_{Ca} M_{Cl} M_{K} M_{Mg} M_{NO3} M_{P} M_{S} m_{NH4} - 2 M_{Ca} M_{Cl} M_{K} M_{NH4} M_{NO3} M_{P} M_{S} m_{Mg} - M_{Ca} M_{Cl} M_{Mg} M_{NH4} M_{NO3} M_{P} M_{S} m_{K} + M_{Ca} M_{K} M_{Mg} M_{NH4} M_{NO3} M_{P} M_{S} m_{Cl}}{2 M_{Cl} M_{K} M_{Mg} M_{NH4} M_{NO3} M_{P} M_{S}}$

    m_Ca = 200.00
    I = 0.000

```python
print(solved_S)
print(solved_Ca)
```

    (-M_Ca*M_Cl*M_K*M_Mg*M_NH4*M_NO3*M_S*m_P - M_Ca*M_Cl*M_K*M_Mg*M_NH4*M_P*M_S*m_NO3 + M_Ca*M_Cl*M_K*M_Mg*M_NO3*M_P*M_S*m_NH4 + 2*M_Ca*M_Cl*M_K*M_NH4*M_NO3*M_P*M_S*m_Mg + M_Ca*M_Cl*M_Mg*M_NH4*M_NO3*M_P*M_S*m_K - M_Ca*M_K*M_Mg*M_NH4*M_NO3*M_P*M_S*m_Cl + 2*M_Cl*M_K*M_Mg*M_NH4*M_NO3*M_P*M_S*m_Ca)/(2*M_Ca*M_Cl*M_K*M_Mg*M_NH4*M_NO3*M_P)
    (2*M_Ca*M_Cl*M_K*M_Mg*M_NH4*M_NO3*M_P*m_S + M_Ca*M_Cl*M_K*M_Mg*M_NH4*M_NO3*M_S*m_P + M_Ca*M_Cl*M_K*M_Mg*M_NH4*M_P*M_S*m_NO3 - M_Ca*M_Cl*M_K*M_Mg*M_NO3*M_P*M_S*m_NH4 - 2*M_Ca*M_Cl*M_K*M_NH4*M_NO3*M_P*M_S*m_Mg - M_Ca*M_Cl*M_Mg*M_NH4*M_NO3*M_P*M_S*m_K + M_Ca*M_K*M_Mg*M_NH4*M_NO3*M_P*M_S*m_Cl)/(2*M_Cl*M_K*M_Mg*M_NH4*M_NO3*M_P*M_S)

## ЕС профиля

Коррекция профиля по EC производится по формуле Зоневельда

$$ EC = 0.095
    \left(
     \frac{m_{NH_4}  r}{M_N} +
     \frac{2  m_{Ca} r}{M_{Ca}} +
     \frac{2  m_{Mg} r}{M_{Mg}} +
     \frac{m_{K} r}{M_{K}}
     \right) + 0.19 $$

где $r$ - это коэфициент приведения

```python
elements = ['NO3', 'NH4', 'P', 'K', 'Ca', 'Mg', 'S', 'Cl']

r = sym.symbols('r')
EC, NH4NO3, KMg, KCa, KN, PN, N = sym.symbols(['EC', 'K_NH4NO3', 'K_KMg', 'K_KCa', 'K_KN', 'K_PN', 'm_N'])

m = dict(zip(elements, sym.symbols([f'm_{e}' for e in elements])))
M = dict(zip(elements, sym.symbols([f'M_{e}' for e in elements])))

repl = {
    r: 1,
    m['NO3'] : 200,
    m['NH4'] : 20,
    m['P'] : 40,
    m['K'] : 180,
    m['Ca'] : 200,
    m['Mg'] : 50,
    m['S']: 72.44,
    m['Cl']: 0.0,
    # Молярные массы
    M['NO3'] : 14.0067, M['NH4'] : 14.0067, M['Ca'] : 40.078,
    M['Mg'] : 24.305, M['K'] : 39.0983, M['P'] : 30.0983, M['S'] : 32.065, M['Cl']: 35.45,
}

# Формулы коэфициентов

NH4NO3f = sym.Eq(m['NH4'] / m['NO3'], NH4NO3)
Nf = sym.Eq(m['NH4'] + m['NO3'], N)

Sum_k = sym.Eq(sum([N, m['K'], m['Ca'], m['Mg']]), 1)
KCaf = sym.Eq(m['K'] / m['Ca'], KCa)
KMgf = sym.Eq(m['K'] / m['Mg'], KMg)
KNf = sym.Eq(m['K'] / N, KN)

# Решаем обратную задачу перевода из коэфициентов в элементы

# Решаем систему уравнений
NH4NO3_s = sym.solve([NH4NO3f, Nf], [m['NH4'], m['NO3']])

print("Решаем систему уравнений для NH3 и NO3")
display({
    Nf,
    NH4NO3f,

})
print("Результат")
display(sym.Eq(m['NH4'], NH4NO3_s[m['NH4']]))
display(sym.Eq(m['NO3'], NH4NO3_s[m['NO3']]))

Elements = [N, m['K'], m['Ca'], m['Mg']]
NKCaMg_s = sym.solve([Sum_k, KNf, KCaf, KMgf], Elements)

print("Решаем систему уравнений для N, K, Ca, Mg")
display({
    Sum_k, KNf, KCaf, KMgf
})
print("Результат")
for e in Elements:
    display(sym.Eq(e, NKCaMg_s[e]))

# Уравнение ионного баланса
Cations = (
    ((m['NH4'] * r) / M['NH4'])
           + ((2*m['Ca']*r) / M['Ca'])
           + ((2*m['Mg']*r) / M['Mg'])
           + ((m['K']*r) / M['K'])
          )
Anions = ((m['NO3']/ M['NO3']) + (m['P'] / M['P']) + ((2*m['S'])/ M['S']) + (m['Cl']/ M['Cl']))

ECf = sym.Eq(EC, 0.095 * Cations + 0.19)

# Вычисляем текущий EC
ECv = ECf.subs(repl).rhs
print('Текущий EC')
print(f'EC = {ECv:0.3f}')

# Вычисляем существующие коэфициенты

r_k = {
    N: repl[m['NO3']] + repl[m['NH4']],
    NH4NO3: repl[m['NH4']] / repl[m['NO3']],
    KCa: repl[m['K']] / repl[m['Ca']],
    KMg: repl[m['K']] / repl[m['Mg']],
}
r_k[KN] = repl[m['K']] / r_k[N]

print(r_k)
print("Вычисляем % элементов по коэфициентам")

rN = NKCaMg_s[N].subs(r_k)
rK = NKCaMg_s[m['K']].subs(r_k)
rCa = NKCaMg_s[m['Ca']].subs(r_k)
rMg = NKCaMg_s[m['Mg']].subs(r_k)

rNH4 = NH4NO3_s[m['NH4']].subs({**r_k, N: rN})
rNO3 = NH4NO3_s[m['NO3']].subs({**r_k, N: rN})

print(f'rN={rN:.4f} rNO3={rNO3:.4f} rNH4={rNH4:.4f} rK={rK:.4f} rCa={rCa:.4f} rMg={rMg:.4f}')

# Решаем уравнение
print("Решаем уравнение с неизвестной r")
display(ECf)
solved_r = sym.solve([ECf],(r))[r]

display(sym.Eq(r, solved_r))

ECnew = ECv
r2 = {**repl,
    EC: ECnew,

    m['NH4'] : rNH4,
    m['NO3']: rNO3,
    m['K'] : rK,
    m['Ca'] : rCa,
    m['Mg'] : rMg,
                  }

# Вычисляем коэфициент
r_k = solved_r.subs(r2)

print(f'r = {r_k:0.2f}')

print('Проверяем новый ЕС')
EC_p = EC.subs({**r2,
                r: r_k
                  })

print(f'EC = {EC_p:0.3f}')

print('Считаем новый профиль')
r3 = dict(repl)
for e in ['NO3', 'NH4', 'K', 'Ca', 'Mg']:

    mE = r2[m[e]] * r_k
    r3[m[e]] = mE
    print(f'{e}={mE:.2f}')

I = (Cations - Anions).subs(r3)
print(f"I = {I:0.3f}")

mS = solved_S.subs(r3)
print(f'S={r3[m["S"]]:0.2f} -> {mS:0.2f}')

# Фосфор - неизменен, сера пересчитывается по балансу

print("Пересчитываем на EC=1.5")

r2 = {**r2,
    EC: 1.5,
    m['NH4'] : rNH4,
    m['K'] : rK,
    m['Ca'] : rCa,
    m['Mg'] : rMg,
                  }

# Вычисляем коэфициент
r_k = solved_r.subs(r2)

print(f'r = {r_k:0.2f}')

print('Проверяем новый ЕС')
EC_p = EC.subs({**r2,
                r: r_k
                  })

print(f'EC = {EC_p:0.3f}')

print('Считаем новый профиль')
r3 = dict(repl)
for e in ['NO3', 'NH4', 'K', 'Ca', 'Mg']:

    mE = r2[m[e]] * r_k
    r3[m[e]] = mE
    print(f'{e}={mE:.2f}')

print(f"NH4/NO3 = {r3[m['NH4']] / r3[m['NO3']]}")
I = (Cations - Anions).subs(r3)
print(f"I = {I:0.3f}")

mS = solved_S.subs(r3)
print(f'S={r3[m["S"]]:0.2f} -> {mS:0.2f}')

I = (Cations - Anions).subs({**r3, m['S']: mS})
print(f"I = {I:0.3f}")

```

    Решаем систему уравнений для NH3 и NO3

$\displaystyle \left\{\frac{m_{NH4}}{m_{NO3}} = K_{NH4NO3}, m_{NH4} + m_{NO3} = m_{N}\right\}$

    Результат

$\displaystyle m_{NH4} = \frac{K_{NH4NO3} m_{N}}{K_{NH4NO3} + 1}$

$\displaystyle m_{NO3} = \frac{m_{N}}{K_{NH4NO3} + 1}$

    Решаем систему уравнений для N, K, Ca, Mg

$\displaystyle \left\{\frac{m_{K}}{m_{Ca}} = K_{KCa}, \frac{m_{K}}{m_{Mg}} = K_{KMg}, \frac{m_{K}}{m_{N}} = K_{KN}, m_{Ca} + m_{K} + m_{Mg} + m_{N} = 1\right\}$

    Результат

$\displaystyle m_{N} = \frac{K_{KCa} K_{KMg}}{K_{KCa} K_{KMg} K_{KN} + K_{KCa} K_{KMg} + K_{KCa} K_{KN} + K_{KMg} K_{KN}}$

$\displaystyle m_{K} = \frac{K_{KCa} K_{KMg} K_{KN}}{K_{KCa} K_{KMg} K_{KN} + K_{KCa} K_{KMg} + K_{KCa} K_{KN} + K_{KMg} K_{KN}}$

$\displaystyle m_{Ca} = \frac{K_{KMg} K_{KN}}{K_{KCa} K_{KMg} K_{KN} + K_{KCa} K_{KMg} + K_{KCa} K_{KN} + K_{KMg} K_{KN}}$

$\displaystyle m_{Mg} = \frac{K_{KCa} K_{KN}}{K_{KCa} K_{KMg} K_{KN} + K_{KCa} K_{KMg} + K_{KCa} K_{KN} + K_{KMg} K_{KN}}$

    Текущий EC
    EC = 2.102
    {m_N: 220, K_NH4NO3: 0.1, K_KCa: 0.9, K_KMg: 3.6, K_KN: 0.8181818181818182}
    Вычисляем % элементов по коэфициентам
    rN=0.3385 rNO3=0.3077 rNH4=0.0308 rK=0.2769 rCa=0.3077 rMg=0.0769
    Решаем уравнение с неизвестной r

$\displaystyle EC = 0.19 + \frac{0.095 m_{NH4} r}{M_{NH4}} + \frac{0.19 m_{Mg} r}{M_{Mg}} + \frac{0.095 m_{K} r}{M_{K}} + \frac{0.19 m_{Ca} r}{M_{Ca}}$

$\displaystyle r = \frac{200.0 EC M_{Ca} M_{K} M_{Mg} M_{NH4} - 38.0 M_{Ca} M_{K} M_{Mg} M_{NH4}}{19.0 M_{Ca} M_{K} M_{Mg} m_{NH4} + 38.0 M_{Ca} M_{K} M_{NH4} m_{Mg} + 19.0 M_{Ca} M_{Mg} M_{NH4} m_{K} + 38.0 M_{K} M_{Mg} M_{NH4} m_{Ca}}$

    r = 650.00
    Проверяем новый ЕС
    EC = 2.102
    Считаем новый профиль
    NO3=200.00
    NH4=20.00
    K=180.00
    Ca=200.00
    Mg=50.00
    I = 0.000
    S=72.44 -> 72.45
    Пересчитываем на EC=1.5
    r = 445.34
    Проверяем новый ЕС
    EC = 1.500
    Считаем новый профиль
    NO3=137.03
    NH4=13.70
    K=123.32
    Ca=137.03
    Mg=34.26
    NH4/NO3 = 0.100000000000000
    I = -1.841
    S=72.44 -> 42.93
    I = 0.000

```python
# Печатаем уравнения для кодирования
for e, f in NH4NO3_s.items():
    print(f'{e} = {f}')
for e, f in NKCaMg_s.items():
    print(f'{e} = {f}')

print(f'r = {solved_r}')
```

    m_NH4 = K_NH4NO3*m_N/(K_NH4NO3 + 1)
    m_NO3 = m_N/(K_NH4NO3 + 1)
    m_N = K_KCa*K_KMg/(K_KCa*K_KMg*K_KN + K_KCa*K_KMg + K_KCa*K_KN + K_KMg*K_KN)
    m_K = K_KCa*K_KMg*K_KN/(K_KCa*K_KMg*K_KN + K_KCa*K_KMg + K_KCa*K_KN + K_KMg*K_KN)
    m_Ca = K_KMg*K_KN/(K_KCa*K_KMg*K_KN + K_KCa*K_KMg + K_KCa*K_KN + K_KMg*K_KN)
    m_Mg = K_KCa*K_KN/(K_KCa*K_KMg*K_KN + K_KCa*K_KMg + K_KCa*K_KN + K_KMg*K_KN)
    r = (200.0*EC*M_Ca*M_K*M_Mg*M_NH4 - 38.0*M_Ca*M_K*M_Mg*M_NH4)/(19.0*M_Ca*M_K*M_Mg*m_NH4 + 38.0*M_Ca*M_K*M_NH4*m_Mg + 19.0*M_Ca*M_Mg*M_NH4*m_K + 38.0*M_K*M_Mg*M_NH4*m_Ca)

## Расчет навески

```python
'''
Решение через уравнение
'''
elements = ['NO3', 'NH4', 'P', 'K', 'Ca', 'Mg', 'S', 'Cl']
Fertilizers = [
    'Ca(NO3)2',

    'KNO3',

    'NH4NO3',

    'MgSO4',

    'KH2PO4',

    'K2SO4',

   # 'Mg(NO3)2',

    'CaCl2',

]

F = dict(zip(Fertilizers, sym.symbols([f'F_{e}' for e in Fertilizers])))

m = dict(zip(elements, sym.symbols([f'm_{e}' for e in elements])))
M = dict(zip(elements, sym.symbols([f'M_{e}' for e in elements])))

F_elems = [
    #['NO3', ['Ca(NO3)2', 'KNO3', 'NH4NO3']],
    ['NH4', ['NH4NO3']],
    ['P', ['KH2PO4']],
    ['K', ['KNO3', 'KH2PO4', 'K2SO4']],
    ['Ca', ['CaCl2', 'Ca(NO3)2']],
    ['Mg', ['MgSO4', 'Mg(NO3)2']],
    ['S', ['MgSO4', 'K2SO4']],
    ['Cl', ['CaCl2']],

]
System = []

for [el, fer] in F_elems:
    for _f in fer:
        if _f in Fertilizers:
            F[f'{_f}_{el}'] = sym.symbols(f'F_{_f}_{el}')

    System.append(
        Eq(m[el],
           sum([F[_f]*F[f'{_f}_{el}'] for _f in fer if _f in Fertilizers])
                    ))

print("Решаем систему уравнений")
display(System)

ferts = sym.solve(System, [F[_f] for _f in Fertilizers])

print("Уравнения по каждому удобрению")
for _f in Fertilizers:
    if _f in Fertilizers:
        display(
            Eq(F[_f], ferts[F[_f]])
        )

repl = {
    #NPK
    m['NO3'] : 200,
    m['NH4'] : 20,
    m['P'] : 40,
    m['K'] : 180,
    m['Ca'] : 200,
    m['Mg'] : 50,
    m['S']: 68.52,
    m['Cl']: 10,
    # Молярные массы
    M['NO3'] : 14.0067, M['NH4'] : 14.0067, M['Ca'] : 40.078,
    M['Mg'] : 24.305, M['K'] : 39.0983, M['P'] : 30.0983, M['S'] : 32.065, M['Cl']: 35.45,
}
repl[m['S']] = solved_S.subs(repl)
ferts_perc ={
    # Процентовки
    'Ca(NO3)2_Ca':16.972,
    'Ca(NO3)2_NO3':11.863,

    'KNO3_K':38.672,
    'KNO3_NO3':13.854,

    'NH4NO3_NO3':17.499,
    'NH4NO3_NH4':17.499,

    'MgSO4_Mg':9.861,
    'MgSO4_S':13.010,

    'KH2PO4_K':28.731,
    'KH2PO4_P':22.761,

    'K2SO4_K':44.874,
    'K2SO4_S':18.401,

    'Mg(NO3)2_Mg': 9.479,
    'Mg(NO3)2_NO3': 10.925,

    'CaCl2_Ca':18.294,
    'CaCl2_Cl':32.366,
}

for [k, v] in ferts_perc.items():
    if k in F:
        repl[F[k]] = v

print("Подставля процентовку удобрений получаем расчет, в г")
weights = {}
for _f in Fertilizers:
    w = ferts[F[_f]].subs(repl)
    display(Eq(F[_f], round(w,4)))
    weights[_f] = w

print("Проверка профиля")

for _e in elements:
    Ep = repl[m[_e]]
    Ef = 0
    for _f in Fertilizers:
        k = f'{_f}_{_e}'
        if k in ferts_perc and _f in weights:
            Ef += weights[_f] * (ferts_perc[k])

    print(f'{_e} = {Ep} - {Ef:0.2f} = {round(Ef - Ep,2)}')

```

    Решаем систему уравнений

$\displaystyle \left[ m_{NH4} = F_{NH4NO3} F_{NH4NO3 NH4}, \  m_{P} = F_{KH2PO4} F_{KH2PO4 P}, \  m_{K} = F_{K2SO4} F_{K2SO4 K} + F_{KH2PO4} F_{KH2PO4 K} + F_{KNO3} F_{KNO3 K}, \  m_{Ca} = F_{Ca(NO3)2} F_{Ca(NO3)2 Ca} + F_{CaCl2} F_{CaCl2 Ca}, \  m_{Mg} = F_{MgSO4} F_{MgSO4 Mg}, \  m_{S} = F_{K2SO4} F_{K2SO4 S} + F_{MgSO4} F_{MgSO4 S}, \  m_{Cl} = F_{CaCl2} F_{CaCl2 Cl}\right]$

    Уравнения по каждому удобрению

$\displaystyle F_{Ca(NO3)2} = \frac{- F_{CaCl2 Ca} m_{Cl} + F_{CaCl2 Cl} m_{Ca}}{F_{Ca(NO3)2 Ca} F_{CaCl2 Cl}}$

$\displaystyle F_{KNO3} = \frac{- F_{K2SO4 K} F_{KH2PO4 P} F_{MgSO4 Mg} m_{S} + F_{K2SO4 K} F_{KH2PO4 P} F_{MgSO4 S} m_{Mg} - F_{K2SO4 S} F_{KH2PO4 K} F_{MgSO4 Mg} m_{P} + F_{K2SO4 S} F_{KH2PO4 P} F_{MgSO4 Mg} m_{K}}{F_{K2SO4 S} F_{KH2PO4 P} F_{KNO3 K} F_{MgSO4 Mg}}$

$\displaystyle F_{NH4NO3} = \frac{m_{NH4}}{F_{NH4NO3 NH4}}$

$\displaystyle F_{MgSO4} = \frac{m_{Mg}}{F_{MgSO4 Mg}}$

$\displaystyle F_{KH2PO4} = \frac{m_{P}}{F_{KH2PO4 P}}$

$\displaystyle F_{K2SO4} = \frac{F_{MgSO4 Mg} m_{S} - F_{MgSO4 S} m_{Mg}}{F_{K2SO4 S} F_{MgSO4 Mg}}$

$\displaystyle F_{CaCl2} = \frac{m_{Cl}}{F_{CaCl2 Cl}}$

    Подставля процентовку удобрений получаем расчет, в г

$\displaystyle F_{Ca(NO3)2} = 11.4511$

$\displaystyle F_{KNO3} = 3.2255$

$\displaystyle F_{NH4NO3} = 1.1429$

$\displaystyle F_{MgSO4} = 5.0705$

$\displaystyle F_{KH2PO4} = 1.7574$

$\displaystyle F_{K2SO4} = 0.1064$

$\displaystyle F_{CaCl2} = 0.309$

    Проверка профиля
    NO3 = 200 - 200.53 = 0.53
    NH4 = 20 - 20.00 = 0
    P = 40 - 40.00 = 0
    K = 180 - 180.00 = 0.0
    Ca = 200 - 200.00 = 0.0
    Mg = 50 - 50.00 = 0
    S = 67.9239233594011 - 67.92 = 0
    Cl = 10 - 10.00 = 0

```python
"""
Алгоритмическое решение
"""

elements = ['NO3', 'NH4', 'P', 'K', 'Ca', 'Mg', 'S', 'Cl']
Fertilizers = [
    'K2SO4',
    'Ca(NO3)2',
    'KNO3',
    'NH4NO3',

    'MgSO4',

    'KH2PO4',

    #'Mg(NO3)2',
    'CaCl2',

]

F_elems = dict([
    ['NO3', ['Ca(NO3)2', 'KNO3', 'NH4NO3', 'Mg(NO3)2']],
    ['NH4', ['NH4NO3']],
    ['P', ['KH2PO4']],
    ['K', ['KNO3', 'KH2PO4', 'K2SO4']],
    ['Ca', ['CaCl2', 'Ca(NO3)2']],
    ['Mg', ['MgSO4', 'Mg(NO3)2']],
    ['S', ['MgSO4', 'K2SO4']],
    ['Cl', ['CaCl2']],
]    )
npk = {
    'NO3' : 200,
    'NH4' : 20,
    'P' : 40,
    'K' : 180,
    'Ca' : 200,
    'Mg' : 56,
    'Cl': 10,
    'S': 67.92,
}

ferts_perc ={
    # Процентовки
    'Ca(NO3)2_Ca':16.972,
    'Ca(NO3)2_NO3':11.863,

    'KNO3_K':38.672,
    'KNO3_NO3':13.854,

    'NH4NO3_NO3':17.499,
    'NH4NO3_NH4':17.499,

    'MgSO4_Mg':9.861,
    'MgSO4_S':13.010,

    'KH2PO4_K':28.731,
    'KH2PO4_P':22.761,

    'K2SO4_K':44.874,
    'K2SO4_S':18.401,

    'Mg(NO3)2_Mg': 9.479,
    'Mg(NO3)2_NO3': 10.925,

    'CaCl2_Ca':18.294,
    'CaCl2_Cl':32.366,
}
for (k, v) in list(ferts_perc.items()):
    f, el = k.split('_')
    if f not in ferts_perc:
        ferts_perc[f] = {}
    ferts_perc[f][el] = v

#print(ferts_perc)

dNPK = dict(sorted(npk.items(), key=lambda i: i[1]))

#print(dNPK)

weights = {}

for el in dNPK:
    for _f in Fertilizers:

        if _f not in F_elems[el]:
            continue
        if el == 'NO3':
            continue
        if _f in weights or _f not in Fertilizers:
            continue
        perc = dict(ferts_perc[_f])
        w = dNPK[el] / perc.pop(el)

        print(f'{_f} = {round(w,4)}')
        for _el, _p in perc.items():
            dNPK[_el] -= w * _p
        weights[_f] = w

print("Проверка профиля")

for _e in elements:
    Ep = npk[_e]
    Ef = 0
    for _f in Fertilizers:
        k = f'{_f}_{_e}'
        if k in ferts_perc and _f in weights:
            Ef += weights[_f] * (ferts_perc[k])

    print(f'{_e} = {Ep:0.2f} - {Ef:0.2f} = {round(Ep - Ef,2)}')

# На данный момент оно собирает неточно

```

    CaCl2 = 0.309
    NH4NO3 = 1.1429
    KH2PO4 = 1.7574
    MgSO4 = 5.6789
    K2SO4 = -0.3241
    KNO3 = 3.7249
    Ca(NO3)2 = 11.4511
    Проверка профиля
    NO3 = 200.00 - 207.45 = -7.45
    NH4 = 20.00 - 20.00 = 0.0
    P = 40.00 - 40.00 = 0.0
    K = 180.00 - 180.00 = 0.0
    Ca = 200.00 - 200.00 = 0.0
    Mg = 56.00 - 56.00 = 0.0
    S = 67.92 - 67.92 = 0.0
    Cl = 10.00 - 10.00 = 0.0

```python
"""
Решение системами линейных уравнений
"""
# Пока не работает
elements = ['NO3', 'NH4', 'P', 'K', 'Ca', 'Mg', 'S', 'Cl']
Fertilizers = [
    'K2SO4',
    'Ca(NO3)2',
    'KNO3',
    'NH4NO3',

    'MgSO4',
    'KH2PO4',

    #'Mg(NO3)2',
    'CaCl2',

]

F_elems = dict([
    ['NO3', ['Ca(NO3)2', 'KNO3', 'NH4NO3', 'Mg(NO3)2']],
    ['NH4', ['NH4NO3']],
    ['P', ['KH2PO4']],
    ['K', ['KNO3', 'KH2PO4', 'K2SO4']],
    ['Ca', ['CaCl2', 'Ca(NO3)2']],
    ['Mg', ['MgSO4', 'Mg(NO3)2']],
    ['S', ['MgSO4', 'K2SO4']],
    ['Cl', ['CaCl2']],
]    )
npk = {
    'NO3' : 200,
    'NH4' : 20,
    'P' : 40,
    'K' : 180,
    'Ca' : 200,
    'Mg' : 56,
    'Cl': 10,
    'S': 67.92,
}

ferts_perc ={
    # Процентовки
    'Ca(NO3)2_Ca':16.972,
    'Ca(NO3)2_NO3':11.863,

    'KNO3_K':38.672,
    'KNO3_NO3':13.854,

    'NH4NO3_NO3':17.499,
    'NH4NO3_NH4':17.499,

    'MgSO4_Mg':9.861,
    'MgSO4_S':13.010,

    'KH2PO4_K':28.731,
    'KH2PO4_P':22.761,

    'K2SO4_K':44.874,
    'K2SO4_S':18.401,

    'Mg(NO3)2_Mg': 9.479,
    'Mg(NO3)2_NO3': 10.925,

    'CaCl2_Ca':18.294,
    'CaCl2_Cl':32.366,
}

for (k, v) in list(ferts_perc.items()):
    f, el = k.split('_')
    if f not in ferts_perc:
        ferts_perc[f] = {}
    ferts_perc[f][el] = v

# Решение

A = []

for el, ferts in F_elems.items():
    r = []
    for f in Fertilizers:
        if f not in ferts:
            r.append(0)
        else:
            r.append(ferts_perc[f][el])
    A.append(r)

B = [[npk[el]] for el in F_elems]

print(A)
print(B)

print(len(A[0]), len(B))

A_len = len(A[0])
B_len = len(B)

if A_len < B_len:
    for i in range(B_len - A_len):
        for r in A:
            r.append(1)

r = [i[0] for i in numpy.linalg.solve(A,B)]

print(r[-1])
weights = dict(zip(Fertilizers, r))

for f, w in weights.items():
    print(f'{f} = {w:.2f}')
# Проверка

print("Проверка профиля")

for _e in elements:
    Ep = npk[_e]
    Ef = 0
    for _f in Fertilizers:
        k = f'{_f}_{_e}'
        if k in ferts_perc and _f in weights:
            Ef += weights[_f] * (ferts_perc[k])

    print(f'{_e} = {Ep:0.2f} - {Ef:0.2f} = {round(Ep - Ef,2)}')

```

    [[0, 11.863, 13.854, 17.499, 0, 0, 0], [0, 0, 0, 17.499, 0, 0, 0], [0, 0, 0, 0, 0, 22.761, 0], [44.874, 0, 38.672, 0, 0, 28.731, 0], [0, 16.972, 0, 0, 0, 0, 18.294], [0, 0, 0, 0, 9.861, 0, 0], [18.401, 0, 0, 0, 13.01, 0, 0], [0, 0, 0, 0, 0, 0, 32.366]]
    [[200], [20], [40], [180], [200], [56], [67.92], [10]]
    7 8
    15.236063909134588
    K2SO4 = -0.06
    Ca(NO3)2 = 11.06
    KNO3 = 3.52
    NH4NO3 = 0.27
    MgSO4 = 4.13
    KH2PO4 = 1.09
    CaCl2 = -0.16
    Проверка профиля
    NO3 = 200.00 - 184.76 = 15.24
    NH4 = 20.00 - 4.76 = 15.24
    P = 40.00 - 24.76 = 15.24
    K = 180.00 - 164.76 = 15.24
    Ca = 200.00 - 184.76 = 15.24
    Mg = 56.00 - 40.76 = 15.24
    S = 67.92 - 52.68 = 15.24
    Cl = 10.00 - -5.24 = 15.24

```python
import numpy.linalg

"""
x - y = -5
2x + y = -7
"""

a = [[1, -1], [2, 1]]
b = [[-5], [-7]]

r = numpy.linalg.solve(a,b)
print(f"x={r[0][0]} y={r[1][0]}")

"""
x - y + z = -5
2x + y + z = -7
"""

a = [[1, -1, 1], [2, 1, 1], [1,1,1]]
b = [[-5], [-7], [0]]

r = numpy.linalg.solve(a,b)
print(f"x={r[0][0]} y={r[1][0]} z={r[2][0]}")

print(-10-4+9)
print(2*-10+4+9)

"""
x - y = -5
2x + y = -7
2x + y = 10
"""

a = [[1, -1, 1], [2, 1, 1], [1, 1, 1]]
b = [[-5], [-7], [10]]

r = numpy.linalg.solve(a,b)
print(f"x={r[0][0]} y={r[1][0]} z={r[2][0]}")

print(-10-4+9)
print(2*-10+4+9)

"""
17.33x = 200
17.33x = 20
"""

a = [[17.33], [17.33]]
b = [200, 20]

w = numpy.linalg.lstsq(a,b)[0][0]
print(w)

```

    x=-4.0 y=1.0
    x=-7.0 y=2.5 z=4.5
    -5
    -7
    x=-17.0 y=7.5 z=19.5
    -5
    -7
    6.347374495095211
    110.0

    <ipython-input-51-5da7e57cab58>:51: FutureWarning: `rcond` parameter will change to the default of machine precision times ``max(M, N)`` where M and N are the input matrix dimensions.
    To use the future default and silence this warning we advise to pass `rcond=None`, to keep using the old, explicitly pass `rcond=-1`.
      w = numpy.linalg.lstsq(a,b)[0][0]

```python
import numpy as np

# https://docs.scipy.org/doc/numpy-1.13.0/reference/generated/numpy.linalg.lstsq.html#numpy.linalg.lstsq
x = np.array([0, 1, 2, 3])
y = np.array([-1, 0.2, 0.9, 2.1])
A = np.vstack([x, np.ones(len(x))]).T
print(A)
r = np.linalg.lstsq(A, y)
print(r)
m, c = r[0]

print(m, c)

```

    [[0. 1.]
     [1. 1.]
     [2. 1.]
     [3. 1.]]
    (array([ 1.  , -0.95]), array([0.05]), 2, array([4.10003045, 1.09075677]))
    1.0000000000000002 -0.9499999999999996

    <ipython-input-38-694e3cc76ac4>:7: FutureWarning: `rcond` parameter will change to the default of machine precision times ``max(M, N)`` where M and N are the input matrix dimensions.
    To use the future default and silence this warning we advise to pass `rcond=None`, to keep using the old, explicitly pass `rcond=-1`.
      r = np.linalg.lstsq(A, y)

```python
import numpy as np

A = [[0, 11.863, 13.854, 17.499, 0, 0, 0], [0, 0, 0, 17.499, 0, 0, 0], [0, 0, 0, 0, 0, 22.761, 0], [44.874, 0, 38.672, 0, 0, 28.731, 0], [0, 16.972, 0, 0, 0, 0, 18.294], [0, 0, 0, 0, 9.861, 0, 0], [18.401, 0, 0, 0, 13.01, 0, 0], [0, 0, 0, 0, 0, 0, 32.366]]
B = [[200], [20], [40], [180], [200], [56], [67.92], [10]]
A_len = len(A[0])
B_len = len(B)

r = np.linalg.qr([A, B])
print(r)
```


# Generative Models

## Generalizing Bayes' Rule

Recall our second problem from [the introduction](./01_bayes_rule_intro.ipynb#Motivating-Examples---Limitations-of-Machine-Learning-and-(frequentist)-Statistics): we observe some heart rate measurements from a workout of a given person. We want to know which heart rate interval contains the true mean for this person with 95% _probability_ a.k.a. the **credible interval** (this is _not_ the same as a 95% confidence interval, see [this thorough explanation](http://jakevdp.github.io/blog/2014/06/12/frequentism-and-bayesianism-3-confidence-credibility/)). We'd also like to know how prior knowledge (e.g. some known population mean and variance) can influence our inference of this interval.

For this, we need to generalize Bayes' rule a bit. Let $y = y_1,\ldots,y_n$ be the data we observe, and $\theta$ a parametric **model** representing some **process that generates the data $y$**. We are now interested in $p(\theta\mid y)$, i.e. the probability (density!) of a model, given the observed data. Bayes' rule can now be written as:

$$
p(\theta\mid y) = \frac{p(y\mid\theta)p(\theta)}{p(y)}
$$

The components of this equation should be interpreted as follows:

- a **likelihood** $p(y\mid\theta)$, i.e. the likelihood of observing $y$, given a particular model $\theta$,
- a **prior distribution** $p(\theta)$, i.e. the distribution of possible parameter values of the model $\theta$,
- a **marginal likelihood** $p(y)$, i.e. the joint distribution $p(y, \theta)$, with $\theta$ integrated out: $\int_\theta p(y\mid\theta)p(\theta)\,\mathrm{d}x$. This component is usually ignored,
- a **posterior distribution** $p(\theta\mid y)$, i.e. the distribution of $\theta$, given its prior and after having observed the data $y$;

To reiterate: the prior and posterior distributions are (continuous) **distributions of model parameters**.

Or, in other words, a model (made up of one or more model parameters) can be seen as just a random variable. It is not fixed at a given value or state.

## Example, observing a single value with a simple model

If we assume that the _process of generating values_ $y_i$ is defined as sampling from a Normal distribution with unknown mean $\mu$ and variance $\sigma^2$: $y \sim \mathcal{N}(\mu, \sigma^2)$, $\theta = \{\mu, \sigma^2\}$. In this case, Bayes' rule looks like:

$$
p(\mu, \sigma^2\mid y_1,\ldots,y_n) = \frac{p(y_1,\ldots,y_n\mid\mu, \sigma^2)p(\mu,\sigma^2)}{p(y_1,\ldots,y_n)}
$$

This seems a bit intimidating, so let's simplify the example by assuming just one observation, $y=170$ bpm. The model $\theta$ can be simplified by considering the variance $\sigma^2$ to be fixed, e.g. at 100 bpm. Also assume we prviously observed a large sample of heart rates with a mean of 130 bpm and variance of 80 from a diverse population. A full specification of the model now looks as follows:

$$
\begin{align}
\mu &\sim\color{red}{\mathcal{N}(m_0, s_0^2)}\,\mathrm{(prior)}\\
y &\sim\color{blue}{\mathcal{N}(\mu, \sigma^2_0)}\,\mathrm{(likelihood)}\\
m_0 &=130\\
s_0^2 &=80\\
\sigma^2_0 &=100
\end{align}
$$

where $\mu$ is a random variable, $y$ is a random variable for which we have observations (n=1), and $m_0$, $s_0^2$ and $\sigma_0^2$ are **hyperparameters** (these are fixed before doing inference). This makes it possible to compute the posterior distribution (note that $\propto$ means "proportional to"):

$$
\begin{align}
p(\mu\mid y) &= \frac{\color{blue}{p(y\mid\mu)}\color{red}{p(\mu)}}{p(y)}\\
&= \frac{\color{blue}{p(y\mid\mu)}\color{red}{p(\mu)}}{\int_{-\infty}^{\infty}p(y\mid\mu)p(\mu)\,\mathrm{d}\mu}\\
&\propto \color{blue}{p(y\mid\mu)}\color{red}{p(\mu)}\,\,(1)\\
&=\color{blue}{\frac{1}{\sqrt{2\pi\color{black}{\sigma_0^2}}}exp\left\{-\frac{1}{2\color{black}{\sigma_0^2}}{(\color{black}{y} - \mu)}^2\right\}}\color{red}{\frac{1}{\sqrt{2\pi \color{black}{s_0^2}}}exp\left\{-\frac{1}{2 \color{black}{s_0^2}}{(\mu - \color{black}{m_0})}^2\right\}}\,\,(2)\\
&\propto \ldots\,\,(3)\\
&\propto exp\left\{-\frac{1}{2 \color{orange}{{\left(\frac{1}{s_0^2} + \frac{1}{\sigma_0^2}\right)}^{-1}}}{\left(\mu - \color{green}{{\left(\frac{1}{s_0^2} + \frac{1}{\sigma_0^2}\right)}^{-1}\left(\frac{m_0}{s_0^2} + \frac{y}{\sigma_0^2}\right)}\right)}^2\right\}\\
&= exp\left\{-\frac{1}{2 \color{orange}{s_1^2}}{(\mu - \color{green}{m1})}^2\right\}\,\,(4)
\end{align}
$$

The result at (4) is, apart from some constant, equal to the normal PDF. The denominator in (1) evaluates to a constant that makes the numerator a proper PDF. Since the normal PDF integrates to 1, we don't need to compute the integral in the denominator in (1).

Be aware that the expression (2) looks very complicated, but actually has only 1 variable, i.e. $\mu$.

For those interested in the rewriting magic hidden in (3) that leads to the result, look [here](http://www.ams.sunysb.edu/~zhu/ams570/Bayesian_Normal.pdf).

Given our likelihood and priors as defined above, the posterior distribution becomes:

$$
\mu\mid y \sim \mathcal{N}(m_1, s_1^2),\\
s_1^2 = {\left(\frac{1}{s_0^2} + \frac{1}{\sigma_0^2}\right)}^{-1}\\
m_1 = s_1^2\left(\frac{m_0}{s_0^2} + \frac{y}{\sigma_0^2}\right)
$$

This can be easily implemented using [scipy's norm object](https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.norm.html):

```python
from scipy.stats import norm  # norm is the scipy object representing a normal distribution

def to_posterior(y, likelihood_variance, prior):
    posterior_variance = 1 / ((1/prior.var()) + (1/likelihood_variance))  # s_1^2 (see above)
    posterior_mean = posterior_variance * ((prior.mean() / prior.var()) + (y / likelihood_variance))  # m_1 (see above)
    posterior = norm(loc=posterior_mean, scale=np.sqrt(posterior_variance))
    setattr(posterior, 'name', f'posterior_{prior.name}')
    return posterior
```

```python
import plotly.graph_objs as go
import numpy as np

# Helper function for creating an x range for plotting multiple density functions
def _dist_range(distributions, n=1000):
    ranges = [dist.ppf([0.001, 0.999]) for dist in distributions]
    return np.linspace(
        min((r[0] for r in ranges)),
        max((r[1] for r in ranges)),
        n
    )

def plot_densities(distributions, title=''):
    x = _dist_range(distributions)
    return go.FigureWidget(
        data=[
            go.Scatter(x=x, y=dist.pdf(x), mode='lines', line={'shape': 'spline', 'width': 4}, fill='tozeroy', name=dist.name)
            for dist in distributions
        ],
        layout=go.Layout(
            title=title
        )
    )
```

What's the influence of the choice of prior on the posterior distribution?

```python
prior_tight = norm(loc=130, scale=1)
setattr(prior_tight, 'name', 'prior_tight')

prior_med = norm(loc=130, scale=10)
setattr(prior_med, 'name', 'prior_med')

prior_flat = norm(loc=130, scale=30)
setattr(prior_flat, 'name', 'prior_flat')

plot_densities([prior_tight, prior_med, prior_flat], title='Prior Densities')
```

How do these priors reflect our (un-)certainty about $\mu$?

```python
plot_densities([to_posterior(170, 100, prior_tight), to_posterior(170, 100, prior_med), to_posterior(170, 100, prior_flat)], title='Posterior Densities')
```

## Multiple Observations

So far, we only considered 1 observation. When dealing with multiple observations, the posterior becomes:

$$
\begin{align}
p(\mu\mid y_1,\ldots,y_n) &\propto \color{blue}{p(y_1,\ldots,y_n\mid\mu)}p(\mu)\\
&= \prod_{i=1}^n \color{blue}{\frac{1}{\sqrt{2\pi\color{black}{\sigma_0^2}}}exp\left\{-\frac{1}{2\color{black}{\sigma_0^2}}{(\color{black}{y_i} - \mu)}^2\right\}}p(\mu)
\end{align}
$$

Because of the [chain rule of probability](https://en.wikipedia.org/wiki/Chain_rule_(probability)), the likelihood of all observations $p(y_1,\ldots,y_n\mid\mu)$
equals the product of likelihoods of each single observation $y_i$. Keep in mind that this is based on the assumption that every $y_i$ is **independent** from the others.

This eventually leads to a similar posterior as for a single observation:

$$
\mu\mid y_1,\ldots,y_n \sim \mathcal{N}(m_1, s_1^2),\\
s_1^2 = {\left(\frac{1}{s_0^2} + \frac{n}{\sigma^2}\right)}^{-1}\\
m_1 = s_1^2\left(\frac{m_0}{s_0^2} + \frac{n\bar{y}}{\sigma^2}\right)
$$

with $n$ being the size, $\bar{y}$ the mean, and $\sigma^2$ the variance of our sample of observations $y_1,\ldots,y_n$. The posterior computation can be generalized as follows:

```python
def to_posterior(y, prior, likelihood_var=1):
    n = len(y)
    y_bar = np.mean(y)
    y_var = np.var(y) if n > 1 else likelihood_var
    posterior_variance = 1 / ((1/prior.var()) + (n/y_var))  # s_1^2 (see above)
    posterior_mean = posterior_variance * ((prior.mean() / prior.var()) + (n*y_bar / y_var))  # m_1 (see above)
    posterior = norm(loc=posterior_mean, scale=np.sqrt(posterior_variance))
    setattr(posterior, 'name', f'posterior_prior={prior.name}_ybar={y_bar:.1f}_yvar={y_var:.1f}_n={n}')
    return posterior
```

Taking the "tight" prior from the previous example, how is it updated based on multiple measurements?

```python
plot_densities([
    prior_tight,
    to_posterior([170], prior_tight, likelihood_var=100),
    to_posterior(norm.rvs(loc=170, scale=10, size=10), prior_tight),
    to_posterior(norm.rvs(loc=170, scale=10, size=100), prior_tight),
    to_posterior(norm.rvs(loc=170, scale=10, size=1000), prior_tight),
], title='Posteriors for "tight" prior')
```

And how about the "medium certainty" prior?

```python
plot_densities([
    prior_med,
    to_posterior([170], prior_med, likelihood_var=100),
    to_posterior(norm.rvs(loc=170, scale=10, size=10), prior_med),
    to_posterior(norm.rvs(loc=170, scale=10, size=100), prior_med),
    to_posterior(norm.rvs(loc=170, scale=10, size=1000), prior_med),
], title='Posteriors for "medium" prior')
```

```python
prior_med.var()
```

```python
norm.var(loc=170, scale=10)
```

```python
to_posterior(norm.rvs(loc=170, scale=10, size=10), prior_med).var()
```

Why is the posterior variance (much) smaller than that of the prior or the sample?

## The Posterior has the Answers

Given that we observe some sample of 10 heart rate measurements from an unknown person's first workout, e.g.

```python
y = norm.rvs(loc=170, scale=10, size=10)
y
```

and given the "medium-certainty" prior defined above (in which a population mean of 130 BPM, with variance of 100 is assumed), the posterior becomes

```python
posterior = to_posterior(y, prior_med)
```

Using the posterior's mean, variance, and (inverse) [Cumulative Distribution Function (CDF)](https://en.wikipedia.org/wiki/Cumulative_distribution_function), it is now possible to answer questions such as:

_What is this user's expected mean heart rate?_

```python
posterior.mean()
```

_What is the probability that this new user's mean heart rate is below 170?_

```python
posterior.cdf(170)
```

_What is the probability that this new user's mean heart rate is between 165 en 175?_

```python
posterior.cdf(175) - posterior.cdf(165)
```

_For which heart rate can we assign a 30% probability that this user's true mean is below it?_

```python
posterior.ppf(0.3)  # This is the inverse of the CDF
```

_What is the range of heart rates for which there is a 95% probability that it contains this user's true mean heart rate?_

```python
f'[{posterior.ppf(0.025)} - {posterior.ppf(0.975)}]'
```

## Introducing ... X

So far we only looked at inference of a single (distribution) parameter based on univariate data $y$. But the Bayesian framework is far more flexible. Can we express a simple linear regression? Assume the most basic example, i.e.

$$
y_i = \beta x_i + \epsilon_i, \epsilon \sim \mathcal{N}(0, \sigma^2)
$$

which could alternatively be written as a likelihood:

$$
y_i\mid\beta,\sigma^2 \sim \mathcal{N}(\beta x_i, \sigma^2)
$$

with PDF:

$$
p(y_i\mid\beta,\sigma^2) = \frac{1}{\sqrt{2\pi\sigma^2}}exp\left\{-\frac{{(y_i - \beta x_i)}^2}{2\sigma^2}\right\}
$$

Be reminded that only $\beta$ and $\sigma^2$ are random variables in this expression, $x$ and $y$ are fixed.

Using similar techniques as above, it is possible to derive the posterior $p(\beta,\sigma^2\mid y)$, **in some cases(*)**.

## * The "Different" Cases

In general, getting the posterior distribution of our parameters of interest involves (depending on the complexity of the model, and number of parameters) a multiplication of 2 or more PDF's, and dividing that by some integral of this product:

$$
\text{posterior} = \frac{\text{product of PDF's of likelihood and (many) different priors}}{\text{the integral over the product of PDF's in the numerator}}
$$

We've seen that in the case of a normally distributed likelihood and normally distributed prior, some clever rewriting gives us a normally distributed posterior (and gives us the value of that nasty integral for free).

But... what if we choose a different prior? There are many different distributions/PDF's to choose from. Instead of the [example model above](#Example,-observing-a-single-value-with-a-simple-model), the prior for $\mu$ could have been a [t distribution](https://en.wikipedia.org/wiki/Student%27s_t-distribution) with PDF

$$
p(\mu) = \frac{\Gamma\left(\frac{\nu+1}{2}\right)}{\Gamma\left(\frac{\nu}{2}\right)\sqrt{\nu\pi}}{\left(1 + \frac{\mu^2}{\nu}\right)}^{-\frac{\nu+1}{2}}
$$

where $\Gamma(x)$ is the [Gamma function](https://en.wikipedia.org/wiki/Gamma_function), or $(x - 1)!$. Taking the product of this PDF and the normal PDF for the likelihood does not result in a well known PDF.

What if, in general, $p(y\mid\theta)p(\theta)$ does not evaluate to a known family of probability density? It means we need to compute that integral in the denominator. Which turns out to be intractable in most cases. Are there other means to get the posterior distribution?


```python
from sympy import *
from sympy.physics.quantum import *
from sympy.physics.quantum.qubit import *
from sympy.physics.quantum.gate import *
from sympy.physics.quantum.gate import IdentityGate as _I
init_printing()
```

```python
# 量子回路図を描画するための準備
%matplotlib inline
import matplotlib.pyplot as plt
from sympy.physics.quantum.circuitplot import *
```

### 4.3.2. アダマールテスト

アダマールテストの例として、制御ユニタリの $ U $ がアダマール演算子の場合を考えています。

制御H演算子は、`CGateS()`を用いて次のようになります。

```python
# 制御H演算子
cH = CGateS(1,H(0))
represent(cH,nqubits=2)
```

アダマールテストの演算子は、次のようになります。

```python
test_gate = H(1)*cH*H(1)
figure_test_gate = CircuitPlot(test_gate,nqubits=2)
```

測定前の量子状態は、次のようになります。

```python
initial_state = Qubit('00')
tested_state = qapply(test_gate*initial_state)
tested_state
```

`measure_partial`関数を使うと、測定後の状態と測定の確率がリストで得られます。１つめの量子ビットが0の場合の量子状態になる確率は、`measure_partial` 関数の要素 [0][1] で得られます。ここで、SymPy では量子ビットの順番を示すインデックスが逆転していることに注意しましょう。つまり、`measure_partial` 関数の第２引数は、`(0,)` ではなく、`(1,)`です。

```python
measured_state0 = measure_partial(tested_state,(1,))[0]
simplify(measured_state0[1])
```

となります。これが $ p_0 = \frac{1+ {\rm Re} \langle \psi | U | \psi \rangle }{2} $ に一致することをアダマールテストは示しています。

つまり、$ p_0 = \frac{\sqrt{2}}{4} + \frac{1}{2}$ ですから、 $ p_0 = \frac{1+ {\rm Re} \langle \psi | U | \psi \rangle }{2} = \frac{\sqrt{2}}{4} + \frac{1}{2} $ から、$ {\rm Re} \langle \psi | U | \psi \rangle  = \frac{\sqrt{2}}{2} $ が得られたことになります。

ここで、$\langle 0 | H | 0 \rangle $ を計算すると、次のように結果が一致します。

```python
qapply(QubitBra('0')*H(0)*Qubit('0'))
```

なお、このときの量子状態は、要素 [0][0] で得られます。

```python
simplify(measured_state0[0])
```

```python

```


# 线性规划 Linear Programming

xyfJASON

## 1 标准型

在一组**线性约束条件**下，最大（小）化**线性目标函数**：

$$
\begin{align}
&\min_x\ c^Tx\\
&\text{s.t.}\begin{cases}
A_{ub}x\leqslant b_{ub}\\
A_{eq}x=b_{eq}\\
l\leqslant x\leqslant u
\end{cases}
\end{align}
$$

> 注：线性规划问题可以求最大、求最小，约束条件可以是小于等于、大于等于，为统一标准，`scipy` 规定标准形式如上，在使用 `scipy` 求解时，首先应化成上述形式。

## 2 变式

### 2.1 含有绝对值

例如：

$$
\begin{align}
&\min_x\ |x_1|+|x_2|+\cdots+|x_n|\\
&\text{s.t.}\ Ax\leqslant b
\end{align}
$$

只需注意到：对任意的 $x_i$，存在 $u_i,v_i\geqslant 0$ 满足：

$$
\begin{cases}
|x_i|=u_i+v_i\\
x_i=u_i-v_i
\end{cases}
$$

（事实上，取 $u_i=(|x_i|+x_i)/2,\,v_i=(|x_i|-x_i)/2$ 即可）

于是原问题转化为：

$$
\begin{align}
&\min_{u,v} \sum_{i=1}^n(u_i+v_i)\\
&\text{s.t.}\begin{cases}A(u-v)\leqslant b\\
u,v\geqslant 0
\end{cases}
\end{align}
$$

进一步改写为：

$$
\begin{align}
&\min \sum_{i=1}^n(u_i+v_i)\\
&\text{s.t.}\begin{cases}[A,-A]\begin{bmatrix}u\\v\end{bmatrix}\leqslant b\\
\begin{bmatrix}u\\v\end{bmatrix}\geqslant 0
\end{cases}
\end{align}
$$

### 2.2 $\min\max$

例如：

$$
\min_x\max_yf(x,y)
$$

只需要引入新的变量：$u=\max\limits_yf(x,y)$，那么一定有：$f(x,y)\leqslant u$，于是问题转化为：

$$
\begin{align}
&\min_x u\\
&\text{s.t.}\ f(x,y)\leqslant u
\end{align}
$$

## 3 算法与代码

求解线性规划的算法有很多，例如单纯形法、改进单纯形法、内点法等等。

我们使用 `scipy.optimize.linprog` 即可。

Documentation: https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.linprog.html

## 4 例题

### 4.1 例一

$$
\begin{align}
&\min z=2x_1+3x_2+x_3\\
&\text{s.t.}\begin{cases}x_1+4x_2+2x_3\geqslant 8\\
3x_1+2x_2\geqslant 6\\
x_1,x_2,x_3\geqslant 0
\end{cases}
\end{align}
$$

编写代码如下：

```python
import numpy as np
from scipy.optimize import linprog

c = np.array([2, 3, 1])
A_ub = np.array([[-1, -4, -2],
                 [-3, -2, 0]])
b_ub = np.array([-8, -6])
res = linprog(c=c,
              A_ub=A_ub,
              b_ub=b_ub,
              bounds=[(0, None), (0, None), (0, None)],
              method='revised simplex')
print(res)
```

         con: array([], dtype=float64)
         fun: 7.0
     message: 'Optimization terminated successfully.'
         nit: 1
       slack: array([0., 0.])
      status: 0
     success: True
           x: array([2., 0., 3.])

### 4.2 例二

$$
\begin{align}
&\min z=|x_1|+2|x_2|+3|x_3|+4|x_4|\\
&\text{s.t.}\begin{cases}x_1-x_2-x_3+x_4\leqslant -2\\
x_1-x_2+x_3-3x_4\leqslant -1\\
x_1-x_2-2x_3+3x_4\leqslant -1/2
\end{cases}
\end{align}
$$

做变量代换：$u_i=(x_i+|x_i|)/2,\,v=(|x_i|-x_i)/2$，则原问题转化为：

$$
\begin{align}
&\min z=u_1+v_1+2u_2+2v_2+3u_3+3v_3+4u_4+4v_4\\
&\text{s.t.}\begin{cases}u_1-v_1-u_2+v_2-u_3+v_3+u_4-v_4\leqslant -2\\
u_1-v_1-u_2+v_2+u_3-v_3-3u_4+3v_4\leqslant -1\\
u_1-v_1-u_2+v_2-2u_3+2v_3+3u_4-3v_4\leqslant -1/2\\
u_1,v_1,u_2,v_2,u_3,v_3,u_4,v_4\geqslant 0
\end{cases}
\end{align}
$$

编写代码如下：

```python
c = np.array([1, 2, 3, 4, 1, 2, 3, 4])
A_ub = np.array([[1, -1, -1, 1, -1, 1, 1, -1],
                 [1, -1, 1, -3, -1, 1, -1, 3],
                 [1, -1, -2, 3, -1, 1, 2, -3]])
b_ub = np.array([-2, -1, -1/2])
res = linprog(c=c,
              A_ub=A_ub,
              b_ub=b_ub,
              bounds=[(0, None)] * 8,
              method='revised simplex')
print(res)
```

         con: array([], dtype=float64)
         fun: 2.0
     message: 'Optimization terminated successfully.'
         nit: 6
       slack: array([0. , 1. , 1.5])
      status: 0
     success: True
           x: array([0., 0., 0., 0., 2., 0., 0., 0.])

### 4.3 例三

$$
\min_{x_i}\max_{y_i}|x_i-y_i|
$$

令 $z=\max\limits_{y_i}|x_i-y_i|$，则 $|x_i-y_i|\leqslant z$，即 $-z\leqslant x_i-y_i\leqslant z$，于是原问题转换为：

$$
\begin{align}
&\min_x z\\
&\text{s.t.}\begin{cases}
x_1-y_1\leqslant z,\ldots,x_n-y_n\leqslant z\\
y_1-x_1\leqslant z,\ldots,y_n-x_n\leqslant z
\end{cases}
\end{align}
$$

成为一个线性规划问题。


# 1 - Imports and defining functions

```python
import numpy as np

from pyFM.mesh import TriMesh
from pyFM.functional import FunctionalMapping

import meshplot as mp

def plot_mesh(myMesh,cmap=None):
    mp.plot(myMesh.vertlist, myMesh.facelist,c=cmap)

def double_plot(myMesh1,myMesh2,cmap1=None,cmap2=None):
    d = mp.subplot(myMesh1.vertlist, myMesh1.facelist, c=cmap1, s=[2, 2, 0])
    mp.subplot(myMesh2.vertlist, myMesh2.facelist, c=cmap2, s=[2, 2, 1], data=d)

def visu(vertices):
    min_coord,max_coord = np.min(vertices,axis=0,keepdims=True),np.max(vertices,axis=0,keepdims=True)
    cmap = (vertices-min_coord)/(max_coord-min_coord)
    return cmap
```

## 2- Loading Data

```python
mesh1 = TriMesh('data/cat-00.off')
mesh2 = TriMesh('data/lion-00.off')
print(f'Mesh 1 : {mesh1.n_vertices:4d} vertices, {mesh1.n_faces:5d} faces\n'
      f'Mesh 2 : {mesh2.n_vertices:4d} vertices, {mesh2.n_faces:5d} faces')

double_plot(mesh1,mesh2)
```

# 3 - Computing the functional map

**Computing descriptors**

```python
process_params = {
    'n_ev': (35,35),  # Number of eigenvalues on source and Target
    'landmarks': np.loadtxt('data/landmarks.txt',dtype=int)[:5],  # loading 5 landmarks
    'subsample_step': 5,  # In order not to use too many descriptors
    'descr_type': 'WKS',  # WKS or HKS
}

model = FunctionalMapping(mesh1,mesh2)
model.preprocess(**process_params,verbose=True);
```

**Fitting the model**

$\newcommand{\RR}{\mathbb{R}}$
$\newcommand{\uargmin}[1]{\underset{#1}{\text{argmin}}\;}$
$\newcommand{\uargmax}[1]{\underset{#1}{\text{argmax}}\;}$
$\def\*#1{\mathbf{#1}}$

Optimization problem is
\begin{equation}
\uargmin{\*C\in\RR^{k_2\times k_1}} \mu_{descr}\|\*C\*A - \*B\|^2 + \mu_{lap}\|\*C\Delta_1 - \Delta_2\*C\|^2 + \mu_{\text{descr comm}}\sum_i \|\*C\Gamma_1^i - \Gamma_2^i\*C\|^2 + \mu_{\text{orient}}\sum_i \|\*C\Lambda_1^i - \Lambda_2^i\*C\|^2
\end{equation}

with $\Gamma_1^i$ and $\Gamma_2^i$ [multipliative operators](http://www.lix.polytechnique.fr/~maks/papers/fundescEG17.pdf) associated to the $i$-th descriptors, $\Lambda_1^i$ and $\Lambda_2^i$ [orientation preserving operators](https://arxiv.org/abs/1806.04455) associated to the $i$-th descriptors

```python
fit_params = {
    'descr_mu': 1e0,
    'lap_mu': 1e-2,
    'descr_comm_mu': 1e-1,
    'orient_mu': 0
}

model.fit(**fit_params, verbose=True)
```

**Visualizing the associated point to point map**

```python
p2p = model.p2p
cmap1 = visu(mesh1.vertlist); cmap2 = cmap1[p2p]
double_plot(mesh1,mesh2,cmap1,cmap2)
```

# 4 - Refining the Functional Map
```model.FM``` returns the current state of functional map. One can change which one is returned by using ```model.change_FM_type(FM_type)```, as one can see below.

**ICP**

```python
model.icp_refine(verbose=True)
p2p = model.p2p
cmap1 = visu(mesh1.vertlist); cmap2 = cmap1[p2p]
double_plot(mesh1,mesh2,cmap1,cmap2)
```

**Zoomout**

```python
model.change_FM_type('classic') # We refine the first computed map, not the icp-refined one

model.zoomout_refine(nit=15, step = 1, verbose=True)
print(model.FM.shape)
p2p = model.p2p
cmap1 = visu(mesh1.vertlist); cmap2 = cmap1[p2p]
double_plot(mesh1,mesh2,cmap1,cmap2)
```

# Evaluating Results

```python
import pyFM.eval
```

```python
# Compute geodesic distance matrix on the cat mesh
A_geod = mesh1.get_geodesic(verbose=True)
```

```python
# Load an approximate ground truth map
gt_p2p = np.loadtxt('data/lion2cat',dtype=int)

model.change_FM_type('classic')
acc_base = pyFM.eval.accuracy(model.p2p, gt_p2p, A_geod, sqrt_area=np.sqrt(mesh1.area))

model.change_FM_type('icp')
acc_icp = pyFM.eval.accuracy(model.p2p, gt_p2p, A_geod, sqrt_area=np.sqrt(mesh1.area))

model.change_FM_type('zoomout')
acc_zo = pyFM.eval.accuracy(model.p2p, gt_p2p, A_geod, sqrt_area=np.sqrt(mesh1.area))

print(f'Accuracy results\n'
      f'\tBasic FM : {1e3*acc_base:.2f}\n'
      f'\tICP refined : {1e3*acc_icp:.2f}\n'
      f'\tZoomOut refined : {1e3*acc_zo:.2f}\n')
```


# Jastrow Factor Derivatives

This notebook calculates the expressions for the derivatives of the following Jastrow function:

$$J(X) = \prod_{i < j} \exp\left(-\frac{\beta^2}{2}r_{ij}^2 + |\beta\gamma|r_{ij}\right)$$

with $N$ particles in $D$ dimensions, $\mathbf{X}\in\mathbb{R}^{N\times D}$ and $r_{ij} = ||\mathbf{X_i} - \mathbf{X_j}||$.

```python
%reload_ext blackcellmagic
from sympy import *
init_printing()
```

```python
N, D, i, j, k, l, d = symbols("N D i j k l d", integer=True)
X = IndexedBase("X", shape=(N, D))

```

## Definition

```python
def r(i, j):
    return sqrt(Sum((X[i, d] - X[j, d]) ** 2, (d, 1, D)))

# r = Function('r', positive=True)

def J(X):
    beta, gamma = symbols(r"\beta \gamma", real=True)
    exponent = Sum(
        -beta ** 2 / 2 * r(i, j) ** 2 + Abs(beta * gamma) * r(i, j),
        (i, 1, N - 1),
        (j, i + 1, N),
    )
    return exp(exponent), (beta, gamma)

Jastrow, (beta, gamma) = J(X)
Jastrow

```

```python
log_Jastrow = expand_log(log(Jastrow), force=True)
log_Jastrow

```

## Derivative w.r.t. $\gamma$

```python
dlogJ_dGamma = diff(log_Jastrow, gamma, 1)
dlogJ_dGamma

```

## Derivative w.r.t. $\beta$

```python
dlogJ_dBeta = diff(log_Jastrow, beta, 1)
dlogJ_dBeta

```

## Derivative w.r.t. $X_{kl}$

```python
dlogJ_dXkl = diff(log_Jastrow, X[k, l], 1)
dlogJ_dXkl

```

```python
# Manually simplify the above:
dlogJ_dXkl = Sum(
    (KroneckerDelta(k, i) - KroneckerDelta(k, j))
    * (X[i, l] - X[j, l])
    * (-beta ** 2 + Abs(beta * gamma) / r(i, j)),
    (i, 1, N - 1),
    (j, i + 1, N),
)
dlogJ_dXkl

```

## Second derivative w.r.t. $X_{kl}$

```python
d2J_dXkl2 = dlogJ_dXkl ** 2 + diff(dlogJ_dXkl, X[k, l], 1)
d2J_dXkl2

```

```python
# Manually simplify the above:
d2J_dXkl2 = dlogJ_dXkl ** 2 + Sum(
    (KroneckerDelta(k, i) - KroneckerDelta(k, j)) ** 2
    * (-beta ** 2 + Abs(beta * gamma) / r(i, j))
    - (KroneckerDelta(k, i) - KroneckerDelta(k, j))
    * (X[i, l] - X[j, l]) ** 2
    * Abs(beta * gamma)
    / r(i, j) ** 3,
    (i, 1, N - 1),
    (j, i + 1, N),
)
d2J_dXkl2

```

```python

```


```python
from sympy import *
from sympy.physics.units import kg, m, s

Newton = kg*m/s**2

a, F, q = var("a, F, q")

sub_list=[
    ( a, 1 *m         ),
    ( F, 1 *Newton    ),
    ( q, 1 *Newton/m  ),
    ]

Ah, Av, MA, Bv, Gh, Gv \
    = var("Ah, Av, MA, Bv, Gh, Gv")

R = 2*a*q

# equilibrium conditions:
eq1 = Eq(  Gh )
eq2 = Eq(  - Bv + F + Gv + R  )
eq3 = Eq(  2*a*F + a*(-Bv + R)  )
eq4 = Eq(  Ah + Gh)
eq5 = Eq(  Av + Gv)
eq6 = Eq(  Gv*a - MA )

unks = [Ah, Av, MA, Bv, Gh, Gv]
eqs = [eq1, eq2, eq3, eq4, eq5, eq6]

sol = solve(eqs, unks)
pprint(sol)

Ah, Av, MA, Bv, Gh, Gv \
    = sol[Ah], sol[Av], sol[MA], sol[Bv], sol[Gh], sol[Gv]

pprint("\nAh, Av, Bv, Gh, Gv, MA in SI units:")
for s in [Ah, Av, Bv, Gh, Gv]:
    tmp = s
    tmp = tmp.subs(sub_list)
    tmp /= Newton
    pprint(tmp)

tmp = MA
tmp = tmp.subs(sub_list)
tmp /= Newton*m
pprint(tmp)

pprint("\nAh, Av, Bv, MA in SI units:")
Ah, Av, MA, Bv= var("Ah, Av, MA, Bv")
eq1 = Eq(  2*F*a + a*(-Bv + R) )
eq2 = Eq(  Ah )
eq3 = Eq(  Av + Bv - R - F )
eq4 = Eq(  2*a*Bv -2*a*R -3*a*F - MA )

unks = [Ah, Av, MA, Bv]
eqs = [eq1, eq2, eq3, eq4]

sol = solve(eqs, unks)
pprint(sol)

# {Ah: 0, Av: -F, Bv: 2⋅F + 2⋅a⋅q, Gh: 0, Gv: F, MA: F⋅a}
#
# Ah, Av, Bv, Gh, Gv, MA in SI units:
# 0
# -1
# 4
# 0
# 1
# 1
#
# Ah, Av, Bv, MA in SI units:
# {Ah: 0, Av: -F, Bv: 2⋅F + 2⋅a⋅q, MA: F⋅a}

```