# Naive Bayes

## 1. Classifying Digits
In this part we will test digits classification on the MNIST dataset, using Bernoulli Naive Bayes (a generative model), in contrast to the Multivariate Logistic Regression (a discriminative model) we used before.

The MNIST dataset contains 28x28 grayscale images of handwritten digits between 0 and 9 (10 classes). For mathmatical analysis clarity, and for matching expected API, each image faltten to create a 1D array with 784 elements.

### Loading the MNIST dataset
Load the MNIST data set. The digits dataset is one of datasets scikit-learn comes with that do not require the downloading of any file from some external website. Use
```
mnist = sklearn.datasets.fetch_mldata('MNIST original')
```
to fetch the original data. You may set the `data_home` to where you wish to download your data for caching. Each image is already transformed into a 1D integer array $x\in [0,255]^{784}$, and the corresponding label is an integer $y\in [0,9]$.

Plot a single sample of each digit as the original image, so you get a feeling how the data looks like.

Finally, divide your data into train and test sets, using 1/7 of the data for testing.

---
**Note 1:** Using `digits = sklearn.datasets.load_digits()` will only fetch a very small sample of the original set, with images resized to 8x8. This preprocessing of the data reduces dimensionality and gives invariance to small distortions - however, we will use the original data in this exercise. Feel free to test the proformance of the algorithms below on the preprocessed data as well.

**Note 2:**
Since ML-Data is deprecated, you may wish to use something like this:
>```
from keras.datasets import mnist
(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train = x_train.reshape(x_train.shape[0], 784)
x_test = x_test.reshape(x_test.shape[0], 784)
```

```python
# your code
from matplotlib import pyplot as plt
import numpy as np
from keras.datasets import mnist
import pandas as pd

# get the mnist data - the data already divided into train and test sets using 1/7 of the data for testing.
(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train = x_train.reshape(x_train.shape[0], 784)
x_test = x_test.reshape(x_test.shape[0], 784)

# plot single sample of each digit
def plot_samples():
    # for each digit - plot the digit - digits in range 0-9
    for i in range(0,10):

        # find the index of the required digit
        j=0
        while y_train[j] != i:
            j+=1

        # plot the digit
        digit_image = x_train[j]
        digit_image = np.array(digit_image, dtype='float')
        pixels = digit_image.reshape((28, 28))
        plt.imshow(pixels, cmap='gray')
        plt.title(f"digit number {i}")
        plt.xlabel("x pixels")
        plt.ylabel("y pixels")
        plt.show()

plot_samples()

```

##### plots explanation
The white pixels are cells with value different than 0 (1-255), and the black pixels are cells with 0 value. we can see that the white pixels present digits between 0-9.

### Bernoulli Naive Bayes
If we know how the digits are generated, then we know how to classify them (simply choose the digit class which will maximize the posterior probability) --- but which model should we use for describing the digits generation?

In this part we will try a very simplified model of digits creation (which is obviously not the same as the "real" model), using a Naive Bayes over an underlying Bernoulli distribution --- that is, we will assume that given a digit class, the pixels of the images are the result of independent coin flips, each with its own "head" probability.

Note that since we assume each pixl is either 0 (black) or 1 (white), we will need to adjust (preprocess) our data accrodingly (see below).

So, the model is stated as follows:
$$
\begin{align}
\text{Domain} && x \in \{0,1\}^{784} \\
\text{Prior} && \pi_j = \Pr(y=j) \\
\text{Likelihood} && P_j(x) = \Pr(x | y=j) \\
\end{align}
$$

Where for each $i\in 0\ldots 784$ it holds that
$$
P_{ji}(x_i) = \Pr(x_i | y=j) =
\begin{cases}
p_{ji} & \text{if } x_i=1 \\
1-p_{ji} & \text{if } x_i=0 \\
\end{cases}
$$

#### Question 1
Write the classification rule based on this Naive Bayes model.
How would you esitmate each of the parameters of the model based on the trainning data?

**Bonus:** Think of edge cases which may effect your estimator in an undesired way.

#### Answer 1
The classification rule is:
$$
c_{NB} = argmax_{j}\pi_j\prod_{i=0}^{N}P_{ji}(x_i)
$$
where N in our case equals 784.

##### Estimations:
prior parameter - we would estimate this parameter from the frequency of classes in the training examples. we would count the number of instances for each class and divide it by the total number of samples.

p_ji(xi) parameter - we would use the independence assumption that the attribute values are conditionally independent given the target value. we would count the number of instances of feature xi given cj class and divide it by the number of instances of cj class.

Likelihood parameter = with the p_ji(xi) parameters $$ \prod_{i=0}^{N}P_{ji}(x_i) $$

##### Bonus
when $$ P_{ji}(x_i) $$ equals to zero it may affect the estimator in an undesired way because it would cause the likelihood to become zero too, and that's not correct since the other probabilities may be greater than zero.

#### Question 2
Run a Naive Bayes classifier on the training data and apply predictions on the test data. Use the [sklearn.naive_bayes.BernoulliNB](http://scikit-learn.org/stable/modules/generated/sklearn.naive_bayes.BernoulliNB.html) implementation (see the [source code for sklearn.naive_bayes](https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/naive_bayes.py) for details).

Remember we need to preprocess the data in this case such that each pixel would become either black (0) or white (1). For this, use the `binarize` parameter of the implementation. Set this value to $0$ (this is the default), which in this case would mean every pixel with non-zero value will be set to 1.

1. Plot the mean image of each class (estimated $\hat{p}_{ji}$) and generate one sample of each class (remember, you can do this since this is a generative model). You will need to access the `feature_log_prob_` attribute of the trained model.

2. Plot the confusion matrix of your classifier, as claculated on the test data (it is recommended to use [sklearn.metrics.confusion_matrix](http://scikit-learn.org/stable/modules/generated/sklearn.metrics.confusion_matrix.html)). Calculate the total accuracy (fraction of correctly classified images), and summarize the results in your own words.

3. Think of a way you can find the optimal threshold of the binarization part. **There is no need to actually perform this task --- just describe what you would have done.**

#### Answer 2

```python
# code goes here
from sklearn.naive_bayes import BernoulliNB
from sklearn.metrics import confusion_matrix

# plot the mean image of each class
def plot_mean_samples():
    print("******Mean Samples******")
    for digit, features in enumerate(nb.feature_log_prob_):
        # plot the digit
        plot_digit(np.exp(features),digit)

# generate samples from the naive bayes distribution
def generate_samples():
    print("******Generated Sampels******")
    # generet each digit
    for digit, class_prob in enumerate(nb.feature_log_prob_):
        sample = []

        # if the value of the pixel in the feature_log_prob_
        # is greater than log(0.5) then the pixel value is 1,
        # else the value is 0
        for value in class_prob:
            if value >= np.log(0.5):
                sample.append(1)
            else:
                sample.append(0)
        plot_digit(sample, digit)

'''
plot the digits
sample = sample to draw - sequence of 1/0
digit - the digit number
'''
def plot_digit(sample,digit):
    # plot the digit
    class_image = np.array(sample, dtype='float')
    pixels = class_image.reshape((28, 28))
    plt.imshow(pixels, cmap='gray')
    plt.title(f"digit number {digit}")
    plt.xlabel("x pixels")
    plt.ylabel("y pixels")
    plt.show()

# plot confusion matrix of the model
# return the confusion matrix
def plot_confusion_matrix():
    print("******Confusion Matrix******")
    cm = confusion_matrix(y_test,y_predict)
    display(cm)
    return cm

# calculate the accuracy from the confusion matrix
def calculate_accuracy(cm):
    true_pred = cm.trace() # sum diagonal values
    total = cm.sum() # sum total values
    accuracy = true_pred / total # calculate accuracy
    print("Accuracy: " + str(accuracy))

nb = BernoulliNB()
nb.fit(x_train,y_train)
y_predict = nb.predict(x_test)

plot_mean_samples()
generate_samples()
cm = plot_confusion_matrix()
calculate_accuracy(cm)

```

##### Summarization of the Results - question 2.2
as we can see from the confusion matrix, the diagonal values present the number of samples that were classified correctly - 8413 samples. the other values present the samples that were classified wrong - 1587. The accuracy of the model is the number of samples that were classified correctly divided by the total number of samples - 84.13%.

##### Answer question 2.3
we can find the optimal threshold of the binarization part with a grid search of the binarize parameter. let's say we are trying the following parameters [0,0.25,0.5,0.75,1], so we choose the best parameter that gives us the best result (the best accuracy) and then we keep searching at the range between this parameter we found and the other one till we got a convergence.
another option can be by calculating the mean image of each class and to take out a certain percent of pixels that are with the lowest probability. For example, if we have 100 pixels, so we will take the 10th highest number and this number will be the threshold.

## 2. Classifing Text Documents using Multinomial Naive Bayes
In this exercise you will classify the "20 newsgroups" data set using your own naive bayes classifier and compare to the scikit learn built in version.

The 20 newsgroups dataset comprises around 18000 newsgroups posts on 20 topics split in two subsets: one for training (or development) and the other one for testing (or for performance evaluation). The split between the train and test set is based upon messages posted before and after a specific date.

### You task
Implement the barebone class provided: `NaiveBayes(BaseEstimator, ClassifierMixin)` and implement its `fit`, `predict` and `predict_proba` methods.

Then follow next anlysis guidlinnes using the implemented class for train and test classifier:
* Load the **train** data using `from sklearn.datasets import fetch_20newsgroups`. remove headers, footers and quotes (see documentation)
* Use `sklearn.feature_extraction.text import CountVectorizer` to count words (stop_words='english')
* use `sklearn.pipeline.make_pipeline` to chain the vectroizer and model.
* note: limit the vocuabolary size if you suffer memory issues
* compare the accuracy over the **test** data. You can use `accuracy_score, classification_report`
* compare to the built in `sklearn.naive_bayes.MultinomialNB`
* compare to `TfidfVectorizer` preprocessing (you can use the built in model for doing the analysis)
* plot the learning curve - is the model in the bias or variance regime (you can use the built in model for doing the analysis)
* optimize performance in respect to vectorizer hyper parameters (e.g. max_features, max_df etc.).

```python
#Classifing Text Documents using Multinomial Naive Bayes

from sklearn.base import BaseEstimator, ClassifierMixin
import pandas as pd
import numpy as np
from sklearn.datasets import fetch_20newsgroups
from sklearn.feature_extraction.text import CountVectorizer, TfidfVectorizer
from sklearn.pipeline import make_pipeline
from sklearn.naive_bayes import MultinomialNB
from sklearn.metrics import accuracy_score
from sklearn.model_selection import learning_curve
import matplotlib.pyplot as plt
from numpy import ma

class NaiveBayes(BaseEstimator, ClassifierMixin):

    # Fit Naive Bayes classifier according to X, y
    def fit(self, x, y):

        self.classes_ = list(set(y)) # get classes
        self.n_features_ = x.get_shape()[1] # get number of features/columns

        # count number of instances from each class and calculate distibutions
        self.class_count_ = pd.Index(y).value_counts()

        self.class_count_.sort_index(inplace=True) #sort according to classes order

        # divide the number of instance for each class by the total number of samples
        self.class_log_prior_ = np.log(np.divide(self.class_count_,len(y)))

        # get class indices
        classes_indices = [[index for index,x in enumerate(y) if x == class_] for class_ in self.classes_]

        # count the total number of times feature  appears in a class
        temp = []
        for indices in classes_indices:
            feature_class = x[indices,:].sum( axis=0 )
            temp.append(feature_class.A1)

        self.feature_count_ = np.array(temp , dtype='float')

        # count the total number of times <b>all<b> features appears in a class
        total_features_class = self.feature_count_.sum(axis=1)

        # smoothed version of maximum likelihood - laplace
        lap_deno = total_features_class[:,None] + 1 * self.n_features_
        lap_nom = self.feature_count_ + 1
        self.feature_log_prob_ = np.log( lap_nom / lap_deno )

    # Return log-probability estimates for the test vector X.
    def predict_log_proba(self, x):
        results = self.class_log_prior_[:, None].T + x.dot(self.feature_log_prob_.T)
        return results

    # Perform classification on an array of test vectors X.
    def predict(self, x):
        # multinomial classifier function
        results = self.class_log_prior_[:, None].T + x.dot(self.feature_log_prob_.T)
        return results.T.argmax( axis=0 ) # get the maximum class index for each sample

# load  data
newsgroups_train = fetch_20newsgroups(subset='train',remove= ('headers', 'footers', 'quotes'))
newsgroups_test = fetch_20newsgroups(subset='test',remove= ('headers', 'footers', 'quotes'))

# Our Naive bayes model
pipline_naive = make_pipeline(CountVectorizer(stop_words='english'), NaiveBayes())
pipline_naive.fit(newsgroups_train['data'],newsgroups_train['target'])
print(f"NaiveBayes Accuracy: { pipline_naive.score(newsgroups_test['data'], newsgroups_test['target']) }")

# Build in MultinomialNB model
pipline_multi = make_pipeline(CountVectorizer(stop_words='english'), MultinomialNB())
pipline_multi.fit(newsgroups_train['data'],newsgroups_train['target'])
print(f"MultinomialNB Accuracy: { pipline_multi.score(newsgroups_test['data'],newsgroups_test['target']) }")

# Build in MultinomialNB model + TfidfVectorizer
pipline_tfidf = make_pipeline(TfidfVectorizer(stop_words='english'), MultinomialNB())
pipline_tfidf.fit(newsgroups_train['data'],newsgroups_train['target'])
print(f"MultinomialNB + TfidfVectorizer Accuracy: {pipline_tfidf.score(newsgroups_test['data'],newsgroups_test['target'])}")

```

    NaiveBayes Accuracy: 0.6343600637280935
    MultinomialNB Accuracy: 0.6343600637280935
    MultinomialNB + TfidfVectorizer Accuracy: 0.6779075942644716

```python
#plot learning curve

# plot the learning curve
def plot_learning_curve(lc):

    # get parameters from lc = learning curve
    train_sizes = lc[0]
    train_scores = lc[1]
    test_scores = lc[2]

    # mean the scores
    train_scores_mean = np.mean(train_scores, axis=1)
    test_scores_mean = np.mean(test_scores, axis=1)

    # Plot learning curve
    plt.figure(figsize=(5, 5))
    plt.title("Learning Curve Naive Bayes")
    plt.ylabel("Score")
    plt.xlabel("Training Examples")
    plt.grid()
    plt.plot(train_sizes, train_scores_mean, 'o-', color="r",
                 label="Training score")
    plt.plot(train_sizes, test_scores_mean, 'o-', color="g",
                 label="Cross-validation score")
    plt.legend()
    plt.show()

lc = learning_curve(
        pipline_multi, newsgroups_train.data, newsgroups_train.target, cv=5)
plot_learning_curve(lc)
```

#### Learning Curve Explanation
As we can see, the model is in the variance regime because the gap between the training score and the cross validation score is wide. (high variance, low bias)

```python
#Optimization -
# min_df=2
pipline_multi_opt1 = make_pipeline(CountVectorizer(stop_words='english',min_df=2),
                                   MultinomialNB())
pipline_multi_opt1.fit(newsgroups_train['data'],newsgroups_train['target'])
print(f"MultinomialNB1 Accuracy: { pipline_multi_opt1.score(newsgroups_test['data'],newsgroups_test['target']) }")

#max_features = 30000
pipline_multi_opt2 = make_pipeline(CountVectorizer(stop_words='english',max_features=30000),
                                   MultinomialNB())
pipline_multi_opt2.fit(newsgroups_train['data'],newsgroups_train['target'])
print(f"MultinomialNB2 Accuracy: { pipline_multi_opt2.score(newsgroups_test['data'],newsgroups_test['target']) }")

# max_df=500
pipline_multi_opt3 = make_pipeline(CountVectorizer(stop_words='english',max_df=500),
                                   MultinomialNB())
pipline_multi_opt3.fit(newsgroups_train['data'],newsgroups_train['target'])
print(f"MultinomialNB3 Accuracy: { pipline_multi_opt3.score(newsgroups_test['data'],newsgroups_test['target']) }")

# max_df=500, min_df=2, max_features=30000
pipline_multi_opt4 = make_pipeline(CountVectorizer(stop_words='english',min_df=2,max_features=30000,max_df=500),
                                   MultinomialNB())
pipline_multi_opt4.fit(newsgroups_train['data'],newsgroups_train['target'])
print(f"MultinomialNB4 Accuracy: { pipline_multi_opt4.score(newsgroups_test['data'],newsgroups_test['target']) }")

```

    MultinomialNB1 Accuracy: 0.6510886882634095
    MultinomialNB2 Accuracy: 0.6524163568773235
    MultinomialNB3 Accuracy: 0.6468401486988847
    MultinomialNB4 Accuracy: 0.659984067976633

### Bonus I (10 points)
#### Model interpretability
Find the most important features for a **specific** decision of a NB classifier.
Because the model has learned the prior $p(x_i|c)$ during the training, the contribution of an individual feature value can be easily measured by the posterior, $p(c|x_i)=p(c)p(x_i|c)/p(x_i)$
Implement a function which gets a scikit-learn NB model as input and returns $P(c|x_i)$:

`def calc_p_c_given_xi(model)`

Hint: Use the following model properties:

* `model.class_log_prior_`
* `model.feature_log_prob_`

Note: remember these are logs and you need to use np.exp and normalize to get $P(c|x_i)$
Another hint: use numpy built-in broadcasting property.

* Use the interpretation to examine errors of the classifier where $\hat{c}\ne c$. Which top words support the correct class and which support the wrong class? You can use the `print_txt` below to color words.

```python
import string
def print_txt(txt, hot, cold):
  """
  print the text, coloring hot and cold words with colors
  """
  cold_color='\x1b[41;37m{}\x1b[0m'
  hot_color='\x1b[42;37m{}\x1b[0m'
  mid_color='\x1b[45;37m{}\x1b[0m'

  def color(token):
    lower = str(token).lower()
    lower = lower.replace('\t','').replace('\n','')
    lower = lower.translate(string.punctuation)
    if (lower in hot) and (lower in cold):
      return mid_color.format(token)
    elif lower in hot:
      return hot_color.format(token)
    elif lower in cold:
      return cold_color.format(token)
    else:
      return token
  colored_txt = " ".join([color(token) for token in txt.split(' ')])
  print(colored_txt)
print_txt('This word support the first class but this the other', ['word'],['other'])
```

    This word support the first class but this the other

```python
# Implementationn for 1st bonus

# calc the posterior - p(c) * p(c|xi) / p(xi)
def calc_p_c_given_xi(model):

    class_prior = np.exp(model.class_log_prior_)
    feature_prob = np.exp(model.feature_log_prob_)
    nominator = class_prior.reshape(-1,1) * feature_prob
    p_xi  = nominator.sum(axis=0)
    p_c_given_xi = nominator / p_xi
    return p_c_given_xi

# intepretation only for the 10 first docs
def interpretation(model):
    index = 0
    for prediction, expected, text in zip(predictions, newsgroups_test['target'], newsgroups_test['data']):

        if index >=10: # iterate only over 10 docs
            break

        if prediction != expected:
            print(f"Document number {index}:\n")
            words_indices = model[index,:].nonzero()[1] # get non zero words indices of the current document
            hot = word_scores(expected, words_indices).word.tolist() # get top 20 words that support correct class
            cold = word_scores(prediction, words_indices).word.tolist() # get top 20 words that support wrong class
            print_txt(text, hot, cold)

        index+=1 # next doc

# rate the words and return top score words
def word_scores(class_, word_indices, min_p=0.1, max_n = 20):

    word_score = p_c_given_xi[class_, word_indices] # distributions of c given xi
    words =[pipline_multi['countvectorizer'].get_feature_names()[word_index] for word_index in word_indices] # get words by indices

    # create series while the indices are (word,index) and the value is word_score
    s = pd.Series(word_score.flatten(), index=[words, word_indices])
    s.index.names=['word','idx']
    s.name = 'score'

    stop = s.sort_values(ascending=False) # sort in order to get top 20
    stop = stop[stop>min_p] # get only words with score greater than min_p=0.1
    if len(stop)>max_n: # if there are more than 20 words take only the top max_n=20
        stop = stop[:max_n]
    return stop.reset_index() # remove indices of (word,index) in order to use regular indices

p_c_given_xi = calc_p_c_given_xi(pipline_multi["multinomialnb"])
predictions = pipline_multi.predict(newsgroups_test['data'])
interpretation(pipline_multi['countvectorizer'].transform(newsgroups_test['data']))

```

    Document number 1:

    I'm not familiar at all with the format of these "X-Face:" thingies, but
    after seeing them in some folks' headers, I've *got* to *see* them (and
    maybe make one of my own)!

    I've got "dpg-view" on my Linux box (which displays "uncompressed X-Faces")
    and I've managed to compile [un]compface too... but now that I'm *looking*
    for them, I can't seem to find any X-Face:'s in anyones news headers!  :-(

    Could you, would you, please send me your "X-Face:" header?

    I *know* I'll probably get a little swamped, but I can handle it.

    	...I hope.
    Document number 2:

    In a word, yes.

    Document number 4:

    I've just spent two solid months arguing that no such thing as an
    objective moral system exists.
    Document number 7:

    A friend of mine managed to get a copy of a computerised Greek and Hebrew
    Lexicon called "The Word Perfect" (That is not the word processing
    package WordPerfect). However, some one wiped out the EXE file, and she
    has not been able to restore it. There are no distributors of the package in
    South Africa. I would appreciate it, if some one could email me the file, or
    at least tell me where I could get it from.

    My email address is
    	fortmann@superbowl.und.ac.za     or
    	fortmann@shrike.und.ac.za

    Many thanks.

### Bonus II (5 points)
#### How can you correct the analyzed error?

To read more about model interpretation, see the blogpost below and the tutorial:
* https://lilianweng.github.io/lil-log/2017/08/01/how-to-explain-the-prediction-of-a-machine-learning-model.html
* https://github.com/chanansh/right_but_why

##### Answer for 2nd bonus

we can find an explanation for the classification - find the features that affect the classification at most and if the classification was wrong we can remove those features and the prediction should move towards the opposite class (the correct class).

```python

```


# Introduction

Molecular energy levels are determined by electronic, vibrational and rotational levels. Spectral lines are dense and they form so called band spectra. Within single band, referent point is determined by electronic or vibrational level. Selection rules for rotational spectra is $\Delta J = 0, \pm 1$, with forbidden $J = 0 \to 0$.

$R$-branch of spectra is branch with one of rotational level being $J' = 1$. Fortan's parabola is described by:

\begin{equation}
k = k_0 + (B' + B'') (J'' + 1) + (B' - B'')(J'' + 1)^2
\end{equation}

# Analysis

```python
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

from scipy.optimize import curve_fit
from scipy.constants import hbar
```

```python
data = pd.read_csv("../data/cn.csv", sep=" ")
```

```python
data
```

      J
      λ
      k

      0
      0
      3878.40
      0.000258

      1
      1
      3878.00
      0.000258

      2
      2
      3877.44
      0.000258

      3
      3
      3876.96
      0.000258

      4
      4
      3876.40
      0.000258

      5
      5
      3875.84
      0.000258

      6
      6
      3875.28
      0.000258

      7
      7
      3874.72
      0.000258

      8
      8
      3874.08
      0.000258

      9
      9
      3873.36
      0.000258

```python
def parabola(x, *args):
    return args[0] + args[1] * x + args[2] * x ** 2
```

```python
data.columns
```

    Index(['J', 'λ', 'k'], dtype='object')

```python
j = data["J"].values
k = data["k"].values * 10 ** 10
```

```python
popt, pcov = curve_fit(parabola, j + 1, k, p0=[2e-3, 100, 10])
```

```python
j_0 = np.linspace(-1, 10, 200)
```

```python
fig, ax = plt.subplots()

ax.scatter(j, k * 10 ** -6, c='b', edgecolor='k')
ax.plot(j_0, parabola(j_0 + 1, *popt) * 10 ** -6)

ax.grid()

ax.set_ylim((.9999 * k.min() * 10 ** -6, 1.0001 * k.max() * 10 ** -6))

ax.set_title("Fortan curve of CN")
ax.set_ylabel(r"$k[Mm]$")
ax.set_xlabel(r"$J''$")
```

```python
b_1 = (popt[1] + popt[2]) / 2
b_2 = (popt[1] - popt[2]) / 2
```

```python
delta = (np.sqrt(pcov[1, 1]) + np.sqrt(pcov[2, 2])) / 2
```

```python
b_1, b_2, delta
```

    (146.17452560353468, 138.25791460956538, 6.170608799469187)

\begin{equation}
\begin{aligned}
B' = (146 \pm 7) m^{-1}\\
B'' = (138 \pm 7) m^{-1}
\end{aligned}
\end{equation}

Moment of intertia is related to $B$ coefficient:

\begin{equation}
B = \frac{\hbar^2}{2I}
\end{equation}

```python
def moment_of_inertia(b, db):
    return hbar ** 2 / (2 * b), hbar ** 2 / (2 * b ** 2) * db
```

```python
i_1, d_i_1 = moment_of_inertia(b_1, delta)
i_2, d_i_2 = moment_of_inertia(b_2, delta)
```

```python
i_1, d_i_1
```

    (3.80408856145368e-71, 1.6058572623613524e-72)

```python
i_2, d_i_2
```

    (4.021909649039747e-71, 1.795024258909058e-72)

\begin{equation}
\begin{aligned}
I' = (3.8 \pm 0.2) 10 ^ {-71} kg m^{2}\\
I'' = (4.0 \pm 0.2) 10 ^ {-71} kg m^{2}
\end{aligned}
\end{equation}

Physics of Molecules/src/Fortan diagrams.ipynb


# Regresión Lineal Desde una Perspectiva Probabilística

> Al día de hoy todos debemos haber ajustado una línea recta (plano, o hiperplano) $f(x) = x^T w$ a un conjunto de puntos $\{(x_i, y_i)\}_{i=1}^{N}$, con $x_i\in\mathbb{R}^{d}$ y $y_i\in\mathbb{R}$, y parámetros $w\in\mathbb{R}^{d}$. Esto normalmente se hace bajo el enfoque de mínimos cuadrados, es decir:
>
> $$
  \hat{w} = \arg \min_{w} ||y - Xw||^2,
  $$

> donde
>
> $$
  X = \left[\begin{array}{ccc}
  - & x_1^T  & - \\
  - & x_2^T  & - \\
    & \vdots &   \\
  - & x_N^T  & - \\
  \end{array}\right] \in \mathbb{R}^{N \times d}, \qquad y = \left[\begin{array}{ccc}
  y_1 \\
  y_2 \\
  \vdots \\
  y_N \\
  \end{array}\right] \in \mathbb{R}^{N}.
  $$

> Adicionalmente, para evitar el fenómeno de overfitting, una práctica común es incluir un término de regularización penalizando el tamaño de los parámetros $w$:
>
> $$
  \hat{w} = \arg \min_{w} ||y - Xw||^2 + \lambda ||w||^2
  $$

> Pues bien, hoy veremos cómo se explica todo esto desde una perspectiva probabilística.

> **Objetivos:**
> - Explicar la regresión lineal por mínimos cuadrados regularizada desde una perspectiva probabilística.

> **Referencias:**
>
> - Bayesian Methods for Machine Learning course, HSE University, Coursera.

## 1. Distribución normal

Antes de entrar en detalles repasemos un poco la distribución normal.

### 1.1. Distribución normal univariada

Se dice que una VA distribuye normal si su función de densidad de probabilidad es:

$$
\mathcal{N}(x|\mu, \sigma^2) = \frac{1}{\sqrt{2 \pi \sigma^2}} \exp \left\{-\frac{(x - \mu)^2}{2 \sigma^2}\right\}
$$

con parámetros $\mu \in \mathbb{R}$: media de la VA X, y $\sigma^2 \in \mathbb{R}$: varianza de la VA X.

**Ejercicio.** Demostrar que:

1.
$$
\int_{-\infty}^{\infty} \mathcal{N}(x|\mu, \sigma^2) = 1.
$$

2.
$$
\mu = \arg \max_{x} \mathcal{N}(x|\mu, \sigma^2)
$$

¿Cómo luce esta densidad?

```python
# Importar scipy.stats
from scipy import stats
# Importar numpy
import numpy as np
# Importar matplitlib.pyplot
from matplotlib import pyplot as plt
```

```python
# Tres VA normales
X = stats.norm(loc=0, scale=1)
Y = stats.norm(loc=1, scale=1)
Z = stats.norm(loc=0, scale=2)
```

```python
# Vector x para graficar
x = np.linspace(-6, 6, 100)
```

```python
# Graficar
fig, ax = plt.subplots(1, 1)
ax.plot(x, X.pdf(x), label=r"$\mu=0, \sigma=1$")
ax.plot(x, Y.pdf(x), label=r"$\mu=1, \sigma=1$")
ax.axvline(x=0, color='k')
ax.axvline(x=1, color='k')
ax.legend()
plt.show()
```

```python
fig, ax = plt.subplots(1, 1)
ax.plot(x, X.pdf(x), label=r"$\mu=0, \sigma=1$")
ax.plot(x, Z.pdf(x), label=r"$\mu=0, \sigma=2$")
ax.axvline(x=0, color='k')
ax.legend()
plt.show()
```

### 1.2. Distribución normal multivariada

Haciendo una extensión natural de lo anterior a vectores, obtenemos la función de densidad de probabilidad para distribución normal multivariada:

$$
\mathcal{N}(x|\mu, \sigma^2) = \frac{1}{\sqrt{\det(2 \pi \Sigma)}} \exp \left\{-\frac{1}{2}(x - \mu)^T \Sigma^{-1} (x - \mu)\right\},
$$

con parámetros $\mu \in \mathbb{R}^n$: vector de medias de la VA X y $\Sigma \in \mathbb{R}^{n \times n}$: matriz de covarianzas de la VA X (simétrica y definida positiva).

¿Cómo luce esta densidad?

```python
# VA normales
X = stats.multivariate_normal(mean=np.zeros(2),
                              cov=np.eye(2))
Y = stats.multivariate_normal(mean=np.array([2, 3]),
                              cov=np.array([[1, 0.2],
                                            [0.8, 2]]))
```

```python
from mpl_toolkits import mplot3d
```

```python
# Gráfico
x = np.linspace(-3, 3, 100)
y = np.linspace(-3, 3, 100)
x, y = np.meshgrid(x, y)
z = X.pdf(np.dstack([x, y]))
fig = plt.figure(figsize=(8, 4))
ax1 = fig.add_subplot(121, projection='3d')
ax1.plot_surface(x, y, z)
ax2 = fig.add_subplot(122)
ax2.contour(x, y, z)
plt.show()
```

```python
# Gráfico
x = np.linspace(-2, 6, 100)
y = np.linspace(-3, 9, 100)
x, y = np.meshgrid(x, y)
z = Y.pdf(np.dstack([x, y]))
fig = plt.figure(figsize=(8, 4))
ax1 = fig.add_subplot(121, projection='3d')
ax1.plot_surface(x, y, z)
ax2 = fig.add_subplot(122)
ax2.contour(x, y, z)
plt.show()
```

## 2. Regresión lineal desde una perspectiva probabilística

Para ver algunos fundamentos de regresión lineal ver la introducción de este cuaderno.

Desde una perspectiva Bayesiana, podemos representar las VA objetivo $y$ y parámetros $w$, datos $X$ de un modelo lineal $y=Xw$ como:

¿Cuál es la factorización que induce este modelo?

$$
P(y, w, X) = ?
$$

Como no nos interesa modelar los datos, sino los parámetros, estudiamos la distribución posterior $P(w | X, y)$, la cual se puede reescribir como:

$$
P(w | X, y) = \frac{P(y, w | X)}{P(y | X)}.
$$

Por otra parte, usando la definición de probabilidad condicional y la factorización inducida, tenemos que:

$$
P(y, w | X) = P(y | X, w) P(w),
$$

de donde

$$
P(w | X, y) = \frac{P(y, w | X)}{P(y | X)} = \frac{P(y | X, w) P(w)}{P(y | X)}.
$$

Las distribuciones posteriores son en general difíciles de encontrar exactamente, dada la complejidad de modelar la distribución de evidencia $P(y | X)$. Durante el curso estaremos viendo técnicas para encontrarlas tanto de manera exacta (cuando sea posible analíticamente) como de manera aproximada.

Para este ejercicio, una suposición plausible es que la función de verosimilitud $P(y | X, w)$ y la distribución previa $P(w)$ son distribuciones normales de la forma:

$$
P(y | X, w) = \mathcal{N}(y | X w, \sigma^2 I),
$$

$$
P(w) = \mathcal{N}(w | 0, \gamma^2 I),
$$

mientras que la evidencia la podemos pensar como un factor de normalización (para que la distribución posterior integre a 1). En general nos interesa encontrar la **distribución posterior**, sin embargo, para este ejemplo, encontraremos sólo **la moda** de la misma.

De modo que para encontrar el valor más probable de los parámetros según la evidencia de los datos y el objetivo, hacemos:

$$
\arg \max_{w} P(w | X, y) = \arg \max_{w} \frac{P(y, w | X)}{P(y | X)} = \arg \max_{w} P(y, w | X) = \arg \max_{w} P(y | X, w) P(w)
$$

___
**Propiedad de los logaritmos:** dado que el logaritmo es una función creciente,

$$
\arg \max_x f(x) = \arg \max_x \log f(x) = \arg \min_x - \log f(x)
$$
___

Así que el anterior problema lo podemos reescribir como:

$$
\arg \min_w -\log P(y | X, w) - \log P(w)
$$

Desarrollando los logaritmos en el pizarrón:

\begin{align}
-\log P(y | X, w) - \log P(w) & = \dots \\
                              & = \dots \\
                              & = \dots \\
                              & = \log C + \frac{1}{2\sigma^2} (y - X w)^T (y - X w) + \frac{1}{2\gamma^2} w^T w \\
                              & = \log C + \frac{1}{2\sigma^2} ||y - X w||^2 + \frac{1}{2\gamma^2} ||w||^2
\end{align}

Finalmente, podemos resolver el problema

$$
\arg \min_w ||y - X w||^2 + \lambda ||w||^2,
$$

con $\lambda = \frac{\sigma^2}{\gamma^2}$.  Interpretación de $\lambda$.

Se puede demostrar que la solución es:

$$
w^{*}_{reg} = (X^T X + \lambda I)^{-1} X^T y
$$

**Ejemplo:** Usar mínimos cuadrados regularizados para ajustar una recta (plano) a los datos de precios de casas en Portland, OR.

1. Usando scipy.optimize.minimize

```python
# Importar pandas
import pandas as pd
# Importar scipy.optimize.minimize
from scipy.optimize import minimize
```

```python
# Leer datos
data = pd.read_csv("housing_prices.csv", low_memory=False)
X = data[["size", "n_bedrooms"]]
y = data["price"] / 100000  # Precios en cientos de miles de dólares
```

```python
# Buena práctica: normalizar los datos
from sklearn.preprocessing import StandardScaler

scaler = StandardScaler()
Xt = scaler.fit_transform(X)
```

```python
# Función objetivo
def sq_reg(w, X, y, reg_coeff):
    # Dimensiones
    N, d = X.shape
    # Aumentar X para el término de bias
    Xa = np.concatenate((np.ones((N, 1)), X), axis=1)
    # Función objetivo
    return np.linalg.norm(y - Xa.dot(w), 2)**2 + reg_coeff * np.linalg.norm(w, 2)**2
```

```python
# Parámetro inicial
w0 = np.zeros(3)
```

```python
# Solución
lr_ridge = minimize(fun=sq_reg,
                    x0=w0,
                    args=(Xt, y, 0.01))
```

```python
lr_ridge
```

          fun: 19.33470672453699
     hess_inv: array([[ 1.06362733e-02, -4.30742104e-06,  3.46984290e-06],
           [-4.30742104e-06,  1.55694290e-02, -8.73565998e-03],
           [ 3.46984290e-06, -8.73565998e-03,  1.55421031e-02]])
          jac: array([-9.53674316e-07,  1.19209290e-06, -2.38418579e-06])
      message: 'Optimization terminated successfully.'
         nfev: 45
          nit: 7
         njev: 9
       status: 0
      success: True
            x: array([ 3.40340246,  1.09412748, -0.06557336])

2. Usando sklearn.linear_model.Ridge

```python
# Importar sklearn.linear_model.Ridge
from sklearn.linear_model import Ridge
# Importar sklearn.pipeline.Pipeline
from sklearn.pipeline import Pipeline
```

```python
# Scaler-regressor pipeline
pipe = Pipeline([
    ("scaler", StandardScaler()),
    ("regressor", Ridge(alpha=0.01))
])
```

```python
# Aprendizaje
pipe.fit(X=X, y=y)
```

    Pipeline(memory=None,
             steps=[('scaler',
                     StandardScaler(copy=True, with_mean=True, with_std=True)),
                    ('regressor',
                     Ridge(alpha=0.01, copy_X=True, fit_intercept=True,
                           max_iter=None, normalize=False, random_state=None,
                           solver='auto', tol=0.001))],
             verbose=False)

```python
pipe.named_steps["regressor"].intercept_, pipe.named_steps["regressor"].coef_
```

    (3.404126595744681, array([ 1.09412745, -0.06557332]))

```python
# Gráfico
plt3d = plt.figure().gca(projection='3d')

xx1, xx2 = np.meshgrid(np.linspace(500, 4500), np.linspace(1, 5, 5))
yp = pipe.predict(np.array([xx1.ravel(), xx2.ravel()]).T).reshape(xx1.shape)
plt3d.plot_surface(xx1, xx2, yp, alpha=0.2)
plt3d.scatter(X['size'], X['n_bedrooms'], y);
plt3d.view_init(elev=10, azim=110)
```

Created with Jupyter by Esteban Jiménez Rodríguez.


# Solving Max-Cut Problem with QAOA

 Copyright (c) 2021 Institute for Quantum Computing, Baidu Inc. All Rights Reserved.

## Overview

In the [tutorial on Quantum Approximate Optimization Algorithm](./QAOA_EN.ipynb), we talked about how to encode a classical combinatorial optimization problem into a quantum optimization problem and slove it with Quantum Approximate Optimization Algorithm [1] (QAOA). In this tutorial, we will take the Max-Cut Problem as an example to further elaborate on QAOA.

### Max-Cut Problem

The Max-Cut Problem is a common combinatorial optimization problem in graph theory, and it has important applications in statistical physics and circuit design. The maximum cut problem is an NP-hard problem, so there is no efficient algorithm that can solve this problem perfectly.

In graph theory, a graph is represented by a pair of sets $G=(V, E)$, where the elements in the set $V$ are the vertices of the graph, and each element in the set $E$ is a pair of vertices, representing an edge connecting these two vertices. For example, the graph in the figure below is represented by $V=\{0,1,2,3\}$ and $E=\{(0,1),(1,2),(2,3),(3, 0)\}$.

Figure 1: A graph with four vertices and four edges

A cut on a graph refers to a partition of the graph's vertex set $V$ into two disjoint sets. Each cut corresponds to a set of edges, in which the two vertices of each edge are divided into different sets. So we can define the size of this cut as the size of this set of edges, that is, the number of edges being cut. The Max-Cut Problem is to find a cut that maximizes the number of edges being cut. Figure 2 shows a maximum cut of the graph in Figure 1. The size of the maximum cut is $4$, which means that all edges in the graph are cut.

Figure 2: A maximum cut of the graph in Figure 1

Assuming that the input graph $G=(V, E)$ has $n=|V|$ vertices and $m=|E|$ edges, we can describe the Max-Cut Problem as a combinatorial optimization problem with $n$ bits and $m$ clauses. Each bit corresponds to a vertex $v$ in the graph $G$, and its value $z_v$ is $0$ or $1$, corresponding to the vertex belonging to the set $S_{0}$ or $S_{1}$, respectively. Thus, each value $z$ of these $n$ bits corresponds to a distinct cut. Each clause corresponds to an edge $(u,v)$ in the graph $G$. A clause requires that the two vertices connected by its corresponding edge take different values, namely $z_u\neq z_v$, which means the edge is cut. In other words, when the two vertices connected by the edge are divided into different sets, we say that the clause is satisfied. Therefore, for each edge $(u,v)$ in the graph $G$, we have

$$
C_{(u,v)}(z) = z_u+z_v-2z_uz_v,
\tag{1}
$$

where $C_{(u,v)}(z) = 1$ if and only if the edge is cut. Otherwise, the function is equal to $0$. The objective function of the entire combinatorial optimization problem is

$$
C(z) = \sum_{(u,v)\in E}C_{(u,v)}(z) = \sum_{(u,v)\in E}z_u+z_v-2z_uz_v.
\tag{2}
$$

Therefore, to solve the maximum cut problem is to find a value $z$ that maximizes the objective function in equation (2).

### Encoding Max-Cut Problem

Here we take the Max-Cut Problem as an example to further elaborate on QAOA. In order to transform the Max-Cut Problem into a quantum problem, we need to use $n$ qubits, where each qubit corresponds to a vertex in the graph $G$. A qubit being in a quantum state $|0\rangle$ or $|1\rangle$ indicates that its corresponding vertex belongs to the set $S_{0}$ or $S_{1}$, respectively. It is worth noting that $|0\rangle$ and $|1\rangle$ are the two eigenstates of Pauli $Z$ gate, and their eigenvalues are respectively $1$ and $-1$, namely

$$
\begin{align}
Z|0\rangle&=|0\rangle,\tag{3}\\
Z|1\rangle&=-|1\rangle.\tag{4}
\end{align}
$$

Therefore, we can use Pauli $Z$ gate to construct the Hamiltonian $H_C$ of the Max-Cut Problem. Because mapping $f(x):x\to(x+1)/2$ maps $-1$ to $0$ and $1$ to $1$, we can replace $z$ in equation (2) with $(Z+I)/2$ ($I$ is the identity matrix) to get the Hamiltonian corresponding to the objective function of the original problem:

$$
\begin{align}
H_C &= \sum_{(u,v)\in E} \frac{Z_u+I}{2} + \frac{Z_v+I}{2}-2\cdot\frac{Z_u+I}{2} \frac{Z_v+I}{2}\tag{5}\\
&= \sum_{(u,v)\in E} \frac{Z_u+Z_v+2I-(Z_uZ_v+Z_u+Z_v+I)}{2}\tag{6}\\
&= \sum_{(u,v)\in E} \frac{I-Z_uZ_v}{2}.\tag{7}
\end{align}
$$

The expected value of this Hamiltonian for a quantum state $|\psi\rangle$ is

$$
\begin{align}
\langle\psi|H_C|\psi\rangle &= \langle\psi|\sum_{(u,v)\in E} \frac{I-Z_uZ_v}{2}|\psi\rangle\tag{8} \\
&= \langle\psi|\sum_{(u,v)\in E} \frac{I}{2}|\psi\rangle-\langle\psi|\sum_{(u,v)\in E} \frac{Z_uZ_v}{2}|\psi\rangle\tag{9}\\
&= \frac{|E|}{2}-\frac{1}{2}\langle\psi|\sum_{(u,v)\in E} Z_uZ_v|\psi\rangle.\tag{10}
\end{align}
$$

If we define

$$
H_D = -\sum_{(u,v)\in E} Z_uZ_v,
\tag{11}
$$

then finding the quantum state $|\psi\rangle$ that maximizes $\langle\psi|H_C|\psi\rangle$ is equivalent to finding the quantum state $|\psi\rangle$ such that $\langle\psi|H_D|\psi \rangle$ is the largest.

## Paddle Quantum Implementation

Now, let's implement QAOA with Paddle Quantum to solve the Max-Cut Problem. There are many ways to find the parameters $\vec{\gamma},\vec{\beta}$. Here we use the gradient descent method in classical machine learning.

To implement QAOA with Paddle Quantum, the first thing to do is to import the required packages. Among them, the `networkx` package can help us handle graphs conveniently.

```python
from IPython.core.display import HTML
display(HTML("<style>pre { white-space: pre !important; }</style>"))
```

```python
# Import related modules from Paddle Quantum and PaddlePaddle
import paddle
from paddle_quantum.circuit import UAnsatz
from paddle_quantum.utils import pauli_str_to_matrix

# Import additional packages needed
import numpy as np
from numpy import pi as PI
import matplotlib.pyplot as plt
import networkx as nx
```

Next, we generate the graph $G$ in the Max-Cut Problem. For the convenience of computation, the vertices here are labeled starting from $0$.

```python
# n is the number of vertices in the graph G, which is also the number of qubits
n = 4
G = nx.Graph()
V = range(n)
G.add_nodes_from(V)
E = [(0, 1), (1, 2), (2, 3), (3, 0)]
G.add_edges_from(E)

# Print out the generated graph G
pos = nx.circular_layout(G)
options = {
    "with_labels": True,
    "font_size": 20,
    "font_weight": "bold",
    "font_color": "white",
    "node_size": 2000,
    "width": 2
}
nx.draw_networkx(G, pos, **options)
ax = plt.gca()
ax.margins(0.20)
plt.axis("off")
plt.show()
```

### Encoding Hamiltonian

In Paddle Quantum, a Hamiltonian can be input in the form of `list`. Here we construct the Hamiltonian $H_D$ in equation (11).

```python
# Construct the Hamiltonian H_D in the form of list
H_D_list = []
for (u, v) in E:
    H_D_list.append([-1.0,'z'+str(u) +',z' + str(v)])
print(H_D_list)
```

    [[-1.0, 'z0,z1'], [-1.0, 'z1,z2'], [-1.0, 'z2,z3'], [-1.0, 'z3,z0']]

As you can see, in this example, the Hamiltonian $H_D$ is

$$
H_D = -Z_0Z_1-Z_1Z_2-Z_2Z_3-Z_3Z_0.
\tag{12}
$$

We can view the matrix form of the Hamiltonian $H_D$ and get information of its eigenvalues:

```python
# Convert Hamiltonian H_D from list form to matrix form
H_D_matrix = pauli_str_to_matrix(H_D_list, n)
# Take out the elements on the diagonal of H_D
H_D_diag = np.diag(H_D_matrix).real
# Get the maximum eigenvalue of H_D
H_max = np.max(H_D_diag)

print(H_D_diag)
print('H_max:', H_max)
```

    [-4.  0.  0.  0.  0.  4.  0.  0.  0.  0.  4.  0.  0.  0.  0. -4.]
    H_max: 4.0

### Building the QAOA circuit

Earlier we introduced that QAOA needs to apply two unitary transformations $U_C(\gamma)$ and $U_B(\beta)$ alternately on the initial state $|s\rangle = |+\rangle^{\otimes n}$. Here, we use the quantum gates and quantum circuit templates provided in Paddle Quantum to build a quantum circuit to achieve this step. It should be noted that in the Max-Cut Problem, we simplify the problem of maximizing the expected value of the Hamiltonian $H_C$ to the problem of maximizing the expected value of the Hamiltonian $H_D$, so the unitary transformations to be used are $U_D(\gamma)$ and $U_B(\beta)$. By alternately placing two circuit modules with adjustable parameters, we are able to build a QAOA circuit

$$
U_B(\beta_p)U_D(\gamma_p)\cdots U_B(\beta_1)U_D(\gamma_1),
\tag{13}
$$

where $U_D(\gamma) = e^{-i\gamma H_D}$ can be constructed with the circuit in the figure below. Another unitary transformation $U_B(\beta)$ is equivalent to applying a $R_x$ gate to each qubit.

Figure 3: Quantum circuit of unitary transformation $e^{i\gamma Z\otimes Z}$

Therefore, the quantum circuit that realizes a layer of unitary transformation $U_B(\beta)U_D(\gamma)$ is shown in Figure 4.

Figure 4: Quantum circuit of unitary transformation $U_B(\beta)U_D(\gamma)$

In Paddle Quantum, the default initial state of each qubit is $|0\rangle$ (the initial state can be customized by input parameters). We can add a layer of Hadamard gates to change the state of each qubit from $|0\rangle$ to $|+\rangle$ so that we get the initial state $|s\rangle = |+\rangle^{\otimes n}$ required by QAOA. In Paddle Quantum, we can add a layer of Hadamard gates to the quantum circuit by calling `superposition_layer()`.

```python
def circuit_QAOA(p, gamma, beta):
    # Initialize the quantum circuit of n qubits
    cir = UAnsatz(n)
    # Prepare quantum state |s>
    cir.superposition_layer()
    # Build a circuit with p layers
    for layer in range(p):
        # Build the circuit of U_D
        for (u, v) in E:
            cir.cnot([u, v])
            cir.rz(gamma[layer], v)
            cir.cnot([u, v])

        # Build the circuit of U_B, that is, add a layer of R_x gates
        for v in V:
            cir.rx(beta[layer], v)

    return cir
```

After running the constructed QAOA quantum circuit, we obtain the output state

$$
|\vec{\gamma},\vec{\beta}\rangle = U_B(\beta_p)U_D(\gamma_p)\cdots U_B(\beta_1)U_D(\gamma_1)|s\rangle.
\tag{14}
$$

### Calculating the loss function

From the output state of the circuit built in the previous step, we can calculate the objective function of the maximum cut problem

$$
F_p(\vec{\gamma},\vec{\beta}) = \langle\vec{\gamma},\vec{\beta}|H_D|\vec{\gamma},\vec{\beta}\rangle.
\tag{15}
$$

To maximize the objective function is equivalent to minimizing $-F_p$. Therefore, we define $L(\vec{\gamma},\vec{\beta}) = -F_p(\vec{\gamma},\vec{\beta})$ as the loss function, that is, the function to be minimized. Then, we use a classical optimization algorithm to find the optimal parameters $\vec{\gamma},\vec{\beta}$. The following code shows a complete QAOA network built with Paddle Quantum and PaddlePaddle:

```python
class Net(paddle.nn.Layer):
    def __init__(self, p, dtype="float64",):
        super(Net, self).__init__()

        self.p = p
        self.gamma = self.create_parameter(shape=[self.p],
                                           default_initializer=paddle.nn.initializer.Uniform(low=0.0, high=2 * PI),
                                           dtype=dtype, is_bias=False)
        self.beta = self.create_parameter(shape=[self.p],
                                          default_initializer=paddle.nn.initializer.Uniform(low=0.0, high=2 * PI),
                                          dtype=dtype, is_bias=False)

    def forward(self):
        # Define QAOA's quantum circuit
        cir = circuit_QAOA(self.p, self.gamma, self.beta)
        # Run the quantum circuit
        cir.run_state_vector()
        # Calculate the loss function
        loss = -cir.expecval(H_D_list)

        return loss, cir
```

### Training quantum neural network

After defining the quantum neural network for QAOA, we use the gradient descent method to update the parameters in the network to maximize the expected value in equation (15).

```python
p = 4      # Number of layers in the quantum circuit
ITR = 120  # Number of training iterations
LR = 0.1   # Learning rate of the optimization method based on gradient descent
SEED = 1024 # Set global RNG seed
```

Here, we optimize the network defined above in PaddlePaddle.

```python
paddle.seed(SEED)

net = Net(p)
# Use Adam optimizer
opt = paddle.optimizer.Adam(learning_rate=LR, parameters=net.parameters())
# Gradient descent iteration
for itr in range(1, ITR + 1):
    # Run the network defined above
    loss, cir = net()
    # Calculate the gradient and optimize
    loss.backward()
    opt.minimize(loss)
    opt.clear_grad()
    if itr% 10 == 0:
        print("iter:", itr, "loss:", "%.4f"% loss.numpy())
    if itr == ITR:
        print("\nThe trained circuit:")
        print(cir)

gamma_opt = net.gamma.numpy()
print("Optimized parameters gamma:\n", gamma_opt)
beta_opt = net.beta.numpy()
print("Optimized parameters beta:\n", beta_opt)
```

    iter: 10 loss: -3.8886
    iter: 20 loss: -3.9134
    iter: 30 loss: -3.9659
    iter: 40 loss: -3.9906
    iter: 50 loss: -3.9979
    iter: 60 loss: -3.9993
    iter: 70 loss: -3.9998
    iter: 80 loss: -3.9999
    iter: 90 loss: -4.0000
    iter: 100 loss: -4.0000
    iter: 110 loss: -4.0000
    iter: 120 loss: -4.0000

    The trained circuit:
    --H----*-----------------*--------------------------------------------------X----Rz(3.140)----X----Rx(0.824)----*-----------------*--------------------------------------------------X----Rz(0.737)----X----Rx(2.506)----*-----------------*--------------------------------------------------X----Rz(4.999)----X----Rx(4.854)----*-----------------*--------------------------------------------------X----Rz(0.465)----X----Rx(1.900)--
           |                 |                                                  |                 |                 |                 |                                                  |                 |                 |                 |                                                  |                 |                 |                 |                                                  |                 |
    --H----X----Rz(3.140)----X----*-----------------*---------------------------|-----------------|----Rx(0.824)----X----Rz(0.737)----X----*-----------------*---------------------------|-----------------|----Rx(2.506)----X----Rz(4.999)----X----*-----------------*---------------------------|-----------------|----Rx(4.854)----X----Rz(0.465)----X----*-----------------*---------------------------|-----------------|----Rx(1.900)--
                                  |                 |                           |                 |                                        |                 |                           |                 |                                        |                 |                           |                 |                                        |                 |                           |                 |
    --H---------------------------X----Rz(3.140)----X----*-----------------*----|-----------------|----Rx(0.824)---------------------------X----Rz(0.737)----X----*-----------------*----|-----------------|----Rx(2.506)---------------------------X----Rz(4.999)----X----*-----------------*----|-----------------|----Rx(4.854)---------------------------X----Rz(0.465)----X----*-----------------*----|-----------------|----Rx(1.900)--
                                                         |                 |    |                 |                                                               |                 |    |                 |                                                               |                 |    |                 |                                                               |                 |    |                 |
    --H--------------------------------------------------X----Rz(3.140)----X----*-----------------*----Rx(0.824)--------------------------------------------------X----Rz(0.737)----X----*-----------------*----Rx(2.506)--------------------------------------------------X----Rz(4.999)----X----*-----------------*----Rx(4.854)--------------------------------------------------X----Rz(0.465)----X----*-----------------*----Rx(1.900)--

    Optimized parameters gamma:
     [3.14046713 0.73681226 4.99897226 0.46481489]
    Optimized parameters beta:
     [0.82379898 2.50618308 4.85422542 1.90024859]

### Decoding the quantum solution

After obtaining the minimum value of the loss function and the corresponding set of parameters $\vec{\gamma}^*,\vec{\beta}^*$, our task has not been completed. In order to obtain an approximate solution to the Max-Cut Problem, it is necessary to decode the solution to the classical optimization problem from the quantum state $|\vec{\gamma}^*,\vec{\beta}^*\rangle$ output by QAOA. Physically, to decode a quantum state, we need to measure it and then calculate the probability distribution of the measurement results:

$$
p(z)=|\langle z|\vec{\gamma}^*,\vec{\beta}^*\rangle|^2.
\tag{16}
$$

Usually, the greater the probability of a certain bit string, the greater the probability that it corresponds to an optimal solution of the Max-Cut problem.

Paddle Quantum provides a function to view the probability distribution of the measurement results of the state output by the QAOA quantum circuit:

```python
# Repeat the simulated measurement of the circuit output state 1024 times
prob_measure = cir.measure(plot=True)
```

After measurement, we can find the bit string with the highest probability of occurrence. Let the vertices whose bit values are $0$ in the bit string belong to the set $S_0$ and the vertices whose bit values are $1$ belong to the set $S_1$. The set of edges between these two vertex sets is a possible maximum cut of the graph.

The following code selects the bit string with the greatest chance of appearing in the measurement result, then maps it back to the classic solution, and draws the corresponding maximum cut:
- The red vertex belongs to the set $S_0$,
- The blue vertex belongs to the set $S_1$,
- The dashed line indicates the edge being cut.

```python
# Find the most frequent bit string in the measurement results
cut_bitstring = max(prob_measure, key=prob_measure.get)
print("The bit string form of the cut found:", cut_bitstring)

# Draw the cut corresponding to the bit string obtained above on the graph
node_cut = ["blue" if cut_bitstring[v] == "1" else "red" for v in V]

edge_cut = [
    "solid" if cut_bitstring[u] == cut_bitstring[v] else "dashed"
    for (u, v) in E
    ]
nx.draw(
        G,
        pos,
        node_color=node_cut,
        style=edge_cut,
        **options
)
ax = plt.gca()
ax.margins(0.20)
plt.axis("off")
plt.show()
```

As you can see, in this example, QAOA has found a maximum cut on the graph.

_______

## References

[1] Farhi, E., Goldstone, J. & Gutmann, S. A Quantum Approximate Optimization Algorithm. [arXiv:1411.4028 (2014).](https://arxiv.org/abs/1411.4028)


# Realization of Non-Recursive Filters

*This jupyter notebook is part of a [collection of notebooks](../index.ipynb) on various topics of Digital Signal Processing.*

## Quantization Effects

Numbers and numerical operations are represented with a finite numerical resolution in digital processors. The same holds for the amplitude values of signals and the algorithmic operations applied to them. Hence, the intended characteristics of a digital filter may deviate in practice due to the finite numerical resolution. The [double-precision floating point representation](https://en.wikipedia.org/wiki/Double-precision_floating-point_format) used in numerical environments like MATLAB or Python/numpy is assumed to be quasi-continuous. This representation serves therefore as reference for the evaluation of quantization effects.

This section investigates the consequences of quantization in non-recursive filters. We first take a look on the quantization of the filter coefficients, followed by the effects caused by a finite numerical resolution of the operations. The realization of non-recursive filters is subject to both effects.

### Quantization of Filter Coefficients

The output signal $y[k]$ of a non-recursive filter with a finite impulse response (FIR) $h[k]$ of length $N$ is given as

\begin{equation}
y[k] = h[k] * x[k] = \sum_{\kappa = 0}^{N-1} h[\kappa] \; x[k - \kappa]
\end{equation}

where $x[k]$ denotes the input signal. The quantized impulse response $h_Q[k]$ (quantized filter coefficients) is yielded by quantizing the impulse response $h[k]$

\begin{equation}
h_Q[k] = \mathcal{Q} \{ h[k] \} = h[k] + e[k]
\end{equation}

where $e[k] = h_Q[k] - h[k]$ denotes the [quantization error](../quantization/introduction.ipynb#Model-of-the-Quantization-Process). Introducing $h_Q[k]$ into above equation and rearranging results in

\begin{equation}
y_Q[k] = \sum_{\kappa = 0}^{N-1} h[\kappa] \; x[k - \kappa] + \sum_{\kappa = 0}^{N-1} e[\kappa] \; x[k - \kappa]
\end{equation}

The input signal $x[k]$ is filtered by the quantization noise $e[k]$ and superimposed to the desired output of the filter. The overall transfer function $H_Q(e^{j \Omega})$ of the filter with quantized filter coefficients is given as

\begin{equation}
H_Q(e^{j \Omega}) = \sum_{k=0}^{N-1} h[k] \; e^{-j \Omega k} + \sum_{k=0}^{N-1} e[k] \; e^{-j \Omega k}  = H(e^{j \Omega}) + E(e^{j \Omega})
\end{equation}

Hence, the quantization of filter coefficients results in a linear distortion of the desired frequency response. To some extent, this distortion can be incorporated into the design of the filter. However, the magnitude of the quantization error $| E(e^{j \Omega}) |$ cannot get arbitrarily small for a finite quantization step $Q$. This limits the achievable attenuation of a digital filter with quantized coefficients. It is therefore important to normalize the filter coefficients $h[k]$ before quantization in order to keep the relative power of the quantization noise small.

#### Example - Effect of coefficient quantization in a lowpass filter

The coefficients of a digital lowpass filter with a cutoff frequency of $\Omega_0 = \frac{\pi}{2}$ are quantized in the following example.

```python
%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt
import scipy.signal as sig

w = 8  # wordlength of quantized coefficients
A = 1  # attenuation of filter coefficients
N = 256  # number of coefficients for filter
Q = 1/(2**(w-1))  # quantization stepsize

def uniform_midtread_quantizer(x, Q):
    # limiter
    x = np.copy(x)
    idx = np.where(x <= -1)
    x[idx] = -1
    idx = np.where(x > 1 - Q)
    x[idx] = 1 - Q
    # linear uniform quantization
    xQ = Q * np.floor(x/Q + 1/2)

    return xQ

# design lowpass
h = A * sig.firwin(N, .5)
# quantize coefficients
hQ = uniform_midtread_quantizer(h, Q)

# plot frequency response
Om, H = sig.freqz(h)
Om, HQ = sig.freqz(hQ)
Om, E = sig.freqz(hQ-h)

plt.figure(figsize=(10, 4))
plt.plot(Om, 20*np.log10(np.abs(H)), label=r'$| H(e^{j \Omega}) |$ in dB (Designed)')
plt.plot(Om, 20*np.log10(np.abs(HQ)), label=r'$| H_Q(e^{j \Omega}) |$ in dB (Quantized coefficients)')
plt.title('Magnitude of the transfer function w and w/o quantization of coefficients')
plt.xlabel(r'$\Omega$')
plt.axis([0, np.pi, -130, 10])
plt.legend(loc=3)
plt.grid()
```

**Exercise**

* Change the wordlength `w` of the quantized filter coefficients. How does the magnitude response $| H_Q(e^{j \Omega}) |$ of the quantized filter change?
* Change the attenuation `A` of the filter coefficients. What changes?
* Why does the magnitude response of the quantized filter $| H_Q(e^{j \Omega}) |$ deviate more from the magnitude response of the designed filter $| H(e^{j \Omega}) |$ in the frequency ranges with high attenuation?

Solution: The magnitude response of the quantized filter deviates more from the designed filter the shorter the wordlength `w` is chosen. This is due to the increasing quantization error for shorter wordlengths. The quantization error increases also in comparison to the filter coefficients if their amplitude (attenuation `A`) is small. The deviations from the designed filter are generally higher for high attenuations since the quantization error is additive $H_Q(e^{j \Omega}) = H(e^{j \Omega}) + E(e^{j \Omega})$, as derived above.

### Quantization of Signals and Operations

Besides the quantization of filter coefficients $h[k]$, also the quantization of the signals, state variables and operations has to be considered in a practical implementation of filters. The computation of the output signal $y[k] = h[k] * x[k]$ of a non-recursive filter by convolution involves multiplications and additions. In digital signal processors numbers are often represented in [fixed-point arithmetic](https://en.wikipedia.org/wiki/Fixed-point_arithmetic) using [two's complement](https://en.wikipedia.org/wiki/Two's_complement). When multiplying two numbers with a wordlength of $w$-bits in this representation the result would require $2w$-bits. Hence the result has to be requantized to $w$-bits. The rounding operation in the quantizer is often realized as truncation of the $w$ least significant bits. The resulting quantization error is known as [round-off error](https://en.wikipedia.org/wiki/Round-off_error). The addition of two numbers may fall outside the maximum/minimum values of the representation and may suffer from clipping. Similar considerations hold also for other number representations, like e.g. [floating point](https://en.wikipedia.org/wiki/Floating_point).

As for the [quantization noise](../quantization/linear_uniform_quantization_error.ipynb#Model-for-the-Quantization-Error), a statistical model for the round-off error in multipliers is used to quantify the average impact of round-off noise in a non-recursive filter.

#### Model for round-off errors in multipliers

As outlined above, multipliers require a requantization of the result in order to keep the wordlength constant. The multiplication of a quantized signal $x_Q[k]$ with a quantized factor $a_Q$ can be written as

\begin{equation}
y_Q[k] = \mathcal{Q} \{ a_Q \cdot x_Q[k] \} = a_Q \cdot x_Q[k] + e[k]
\end{equation}

where the round-off error $e[k]$ is defined as

\begin{equation}
e[k] = y_Q[k] - a_Q \cdot x_Q[k]
\end{equation}

This leads to the following model of a multiplier including round-off effects

The round-off error can be modeled statistically in the same way as quantization noise [[Zölzer](../index.ipynb#Literature)]. Under the assumption that the average magnitude of $a_Q \cdot x_Q[k]$ is much larger that the quantization step size $Q$, the round-off error $e[k]$ can be approximated by the following statistical model

1. The round-off error $e[k]$ is not correlated with the input signal $x_Q[k]$

2. The round-off error is [white](../random_signals/white_noise.ipynb)

    $$ \Phi_{ee}(\mathrm{e}^{\,\mathrm{j}\,\Omega}) = \sigma_e^2 $$

2. The probability density function (PDF) of the round-off error is given by the zero-mean [uniform distribution](../random_signals/important_distributions.ipynb#Uniform-Distribution)

    $$ p_e(\theta) = \frac{1}{Q} \cdot \text{rect} \left( \frac{\theta}{Q} \right) $$

The variance (power) of the round-off error is [derived from its PDF](../random_signals/important_distributions.ipynb#Uniform-Distribution) as

\begin{equation}
\sigma_e^2 = \frac{Q^2}{12}
\end{equation}

#### Round-off noise

Using above model of a multiplier and discarding clipping, a straightforward realization of the convolution with quantized signals would be to requantize after every multiplication

\begin{equation}
\begin{split}
y_Q[k] &= \sum_{\kappa = 0}^{N-1} \mathcal{Q} \{ h_Q[\kappa] \; x_Q[k - \kappa] \} \\
&= \sum_{\kappa = 0}^{N-1} h_Q[\kappa] \; x_Q[k - \kappa] + e[\kappa] \\
&= \left( \sum_{\kappa = 0}^{N-1} h_Q[\kappa] \; x_Q[k - \kappa] \right) + e_\text{tot}[k]
\end{split}
\end{equation}

where $e_\text{tot}[k] = \sum_{\kappa = 0}^{N-1} e[\kappa]$. The round-off errors $e[\kappa]$ for each multiplication are uncorrelated to each other. The overall power of the round-off error is then given as

\begin{equation}
\sigma_{e, \text{tot}}^2 = N \cdot \frac{Q^2}{12}
\end{equation}

Many digital signal processors allow to perform the multiplications and additions in an internal register with double wordlength. In this case, only the result of the convolution sum has to be requantized

\begin{equation}
y_Q[k] = \mathcal{Q} \left\{ \sum_{\kappa = 0}^{N-1}  h_Q[\kappa] \; x_Q[k - \kappa] \right\} =
\left( \sum_{\kappa = 0}^{N-1} h_Q[\kappa] \; x_Q[k - \kappa] \right) + e[k]
\end{equation}

and the total power of the round-off noise in this case is

\begin{equation}
\sigma_{e, \text{tot}}^2 = \frac{Q^2}{12}
\end{equation}

It is evident that this realization is favorable due to the lower round-off noise, especially for filters with a large number $N$ of coefficients.

#### Example - Effect of requantization after each multiplication

The following example simulates the round-off noise of a non-recursive filter when requantization is performed after each multiplication. Clipping is not considered. The input signal $x[k]$ is drawn from a [uniform distribution](../random_signals/important_distributions.ipynb#Uniform-Distribution) with $a=-1$ and $b=1-Q$. Both the input signal and filter coefficients are quantized. The output signal $y[k]$ without requantization of the multiplications is computed, as well as the output signal $y_Q[k]$ with requantization. The statistical properties of the round-off noise $e[k] = y_Q[k] - y[k]$ are evaluated.

```python
w = 16  # wordlength of quantized coefficients/operations
N = 32  # number of coefficients for filter
L = 8192  # length of input signal
Q = 1/(2**(w-1))  # quantization stepsize

def uniform_midtread_quantizer(x, Q):
    xQ = Q * np.floor(x/Q + 1/2)

    return xQ

# random impulse response
h = np.random.uniform(size=N, low=-1, high=1)
hQ = uniform_midtread_quantizer(h, Q)
# input signal
x = np.random.uniform(size=L, low=-1, high=1-Q)
xQ = uniform_midtread_quantizer(x, Q)
# output signal by convolution
y = np.zeros(L+N-1)
yQ = np.zeros(L+N-1)
for k in np.arange(L):
    for kappa in np.arange(N):
        if (k-kappa) >= 0:
            y[k] += hQ[kappa] * xQ[k-kappa]
            yQ[k] += uniform_midtread_quantizer(hQ[kappa] * xQ[k-kappa], Q)

# overall round-off error
e = yQ - y

# estimate power of round-off error
sx = 10*np.log10(np.var(e))
print('Power of overall round-off noise is %f dB' %sx)
# estimated PDF of round-off error
pe, bins = np.histogram(e, bins=50, density=True, range=(-10*Q, 10*Q))
# estimate PSD of round-off error
nf, Pee = sig.welch(e, nperseg=128)

# plot statistical properties of error signal
plt.figure(figsize=(10,6))

plt.subplot(121)
plt.bar(bins[:-1]/Q, pe*Q, width = 20/len(pe))
plt.title('Estimated PDF of the round-off noise')
plt.xlabel(r'$\theta / Q$')
plt.ylabel(r'$\hat{p}_x(\theta) / Q$')
#plt.axis([-1, 1, 0, 1.2])

plt.subplot(122)
plt.plot(nf*2*np.pi, Pee*6/Q**2/N)
plt.title('Estimated PSD of the round-off noise')
plt.xlabel(r'$\Omega$')
plt.ylabel(r'$\hat{\Phi}_{ee}(e^{j \Omega}) / \sigma_e^2$')
plt.axis([0, np.pi, 0, 2])
plt.grid();
```

**Exercise**

* Change the wordlength `w` and check if the $\sigma_e^2$ derived by numerical simulation is equal to its theoretic value derived above?
* Can you explain the shape of the estimated PDF for the round-off noise?

Solution: The power of the overall round-off noise was derived above as $\sigma_e^2 = N \frac{Q}{12}$. Using results from the [model of quantization noise for a uniform quantizer](../quantization/linear_uniform_quantization_error.ipynb#Model-for-the-Quantization-Error) this can be rewritten as $10 \log_{10} (\sigma_e^2) \approx 10 \log_{10} (N) - 10 \log_{10} (3) - 6.02 w$. Hence a doubling of the filter length results in approximately 3 dB more round-off noise at the filter output. The requantization noise of the individual multiplications can be modeled as uniformly distributed white noise. It follows from the [central limit theorem](https://en.wikipedia.org/wiki/Central_limit_theorem) that the superposition of these individual round-off errors results in a normal distribution.

#### Example - Comparison of round-off noise of conventional and fast convolution

The overall round-off noise in an FIR filter scales with the number of multiplications when rounding is performed after each of these. The fast convolution algorithm requires less multiplications and as a consequence also less rounding operations resulting in potentially less round-off noise at the filter output. This effect is illustrated by the following example. The conventional convolution and the fast convolution is computed for two rectangular signals of the same length $y[k] = A\cdot\text{rect}_L[k] * A\cdot\text{rect}_L[k]$. The round-off noise is computed by comparing the results to the theoretical result $A^2 L \cdot \Lambda_{2L -1}[k]$. Since we cannot change the representation of numbers nor the rounding operations in the given implementation of the FFT, a floating point representation of the signal amplitudes is used for this example. The amplitude $A$ has been chosen such to trigger rounding operations.

```python
L = 1024  # length of signals x[k]
A = 1/7  # amplitude of signal
M = 2*L-1

# generate signals
x = A * np.ones(L)
y = A**2 * L * sig.triang(M)

# linear convolution
y1 = np.convolve(x, x, 'full')
e1 = y - y1
# fast convolution
y2 = np.fft.irfft(np.fft.rfft(x, M+1) * np.fft.rfft(x, M+1))[0:M]
e2 = y - y2

plt.figure(figsize=(10, 4))
plt.plot(np.abs(e1), label='conventional convolution')
plt.plot(np.abs(e2), label='fast convolution')
plt.xlabel(r'$k$')
plt.ylabel(r'$|e[k]|$')
plt.legend()
plt.grid()
```

**Exercises**

* Change the length `L` of the signal. How does the round-off noise scale with length? Why?
* Extend above code to compute the power of the round-off noise for various lengths.

Solution: The round-off noise scales with the number of multiplications which itself scales by the length `L` of the signal. This also explains the shape of the round-off errors for the conventional convolution. The two rectangular functions overlap fully in the center which results in the maximum number of non-zero multiplications.


```python
%matplotlib inline

from __future__ import absolute_import, division, print_function

from itertools import product
from os.path import expanduser, expandvars, join
import os
import sys
from time import time

import numpy as np
import pandas as pd

import matplotlib as mpl
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import axes3d
```

```python
from pisa import ureg
from pisa.core.binning import OneDimBinning
from pisa.core.map import Map
```

    << PISA is running in double precision (FP64) mode; numba is running on GPU >>

```python
retro_path = expanduser(expandvars('~/src/retro'))
if retro_path not in sys.path:
    sys.path.append(retro_path)
```

```python
from retro import *
from retro.table_readers import *
```

```python
def plot_sphere(ax):
    u = np.linspace(0, 2 * np.pi, 100)
    v = np.linspace(0, np.pi, 100)
    x = np.outer(np.cos(u), np.sin(v))
    y = np.outer(np.sin(u), np.sin(v))
    z = np.outer(np.ones(np.size(u)), np.cos(v))
    # Plot the surface
    sph = ax.plot_surface(x, y, z, alpha=0.2)
    return sph
```

```python
def plot_points_on_sph(x, y, z, ax_x, ax_y, ax_z, title=None, ax=None):
    if ax is None:
        fig = plt.figure(figsize=(8, 8), dpi=72)
        ax = fig.add_subplot(111, projection='3d')
    fig = ax.figure

    axlw = 3
    xs, ys, zs = coords2xsyszs(((0, 0, 0), (0, 0, 1)))
    ax.plot(xs, ys, zs, '-', color='k', lw=axlw, alpha=1)
    xs, ys, zs = coords2xsyszs(((0, 0, 0), (0, 1, 0)))
    ax.plot(xs, ys, zs, '-', color='k', lw=axlw, alpha=1)
    xs, ys, zs = coords2xsyszs(((0, 0, 0), (1, 0, 0)))
    ax.plot(xs, ys, zs, '-', color='k', lw=axlw, alpha=1)

    xs, ys, zs = coords2xsyszs(((0, 0, 0), (ax_x, ax_y, ax_z)))
    ax.plot(xs, ys, zs, '-', color='C0', lw=1, alpha=1)

    alpha = 0.25
    xs, ys, zs = coords2xsyszs(((-1, 0, 0), (-1, ax_y, ax_z)))
    #print(xs, ys, zs)
    ax.plot(xs, ys, zs, '-', color='k', lw=2, alpha=alpha)

    xs, ys, zs = coords2xsyszs(((0, -1, 0), (ax_x, -1, ax_z)))
    ax.plot(xs, ys, zs, '-', color='k', lw=2, alpha=alpha)

    xs, ys, zs = coords2xsyszs(((0, 0, -1), (ax_x, ax_y, -1)))
    ax.plot(xs, ys, zs, '-', color='k', lw=2, alpha=alpha)

    sph = plot_sphere(ax)

    pts = ax.scatter(x, y, z, marker='o', color='C2', s=1)

    ax.axis('image')
    ax.view_init(30, 30)
    ax.set_xlim3d(-1, 1)
    ax.set_ylim3d(-1, 1)
    ax.set_zlim3d(-1, 1)
    #ax.set_xlabel('$x$')
    #ax.set_ylabel('$y$')
    #ax.set_zlabel('$z$')
    ax.set_xticks([-1, 0, 1])
    ax.set_yticks([-1, 0, 1])
    ax.set_zticks([-1, 0, 1])
    if title:
        ax.set_title(title)
    #ax.grid(False)
    #ax.set_axis_off()
    fig.tight_layout()

    return fig, ax, pts, sph
```

```python
def coords2xsyszs(coords):
    if len(coords[0]) == 2:
        xs, ys, zs = [c[0] for c in coords], [c[1] for c in coords], [0 for c in coords]
    elif len(coords[0]) == 3:
        xs, ys, zs = [c[0] for c in coords], [c[1] for c in coords], [c[2] for c in coords]
    else:
        raise ValueError()
    return xs, ys, zs
```

```python
outdir = 'cone_in_spherical_coords'
mkdir(outdir)
```

```python
ftype = np.float64
n_samp = int(1e3)
bin_phi = 0
R = 1
n = 1.5 # phase vel. in ice
beta = 1 # approx. vel of muon relative to c
cherenkov_angle = np.arccos(1/(n * beta))
print('Cherenkov angle: {:.3f} deg'.format(np.rad2deg(cherenkov_angle)))
```

    Cherenkov angle: 48.190 deg

Let's start by defining the cone with axis colinear with the $z$-axis and take some number of samples from the circle where that cone intersects the unit sphere.

```python
r = np.full(n_samp, R, dtype=ftype)
phi = np.linspace(0, 2*np.pi * (1 - 1/n_samp), n_samp, dtype=ftype)
theta = np.full(n_samp, cherenkov_angle, dtype=ftype)
#print('r:', r[0])
#print('theta:', np.rad2deg(theta[0]))
#print('phi:', np.rad2deg(phi))
```

```python
x, y, z = np.empty(n_samp, dtype=ftype), np.empty(n_samp, dtype=ftype), np.empty(n_samp, dtype=ftype)
sph2cart(r, theta, phi, x, y, z)
points = np.stack((x, y, z))
```

```python
fig, ax, pts, sph = plot_points_on_sph(x, y, z, 0, 0, 1, title='Axis in $z$ direction')
```

Now rotate the cone to have its axis aligned with an arbitrary (unit) vector defined by $\theta$, $\phi$ angles.

```python
ax_r = np.array([1], dtype=ftype)
ax_x, ax_y, ax_z = np.empty(1, dtype=ftype), np.empty(1, dtype=ftype), np.empty(1, dtype=ftype)

thetas = np.arange(0, 180+1, 15)
phis = np.arange(0, 360, 15)

theta0_computed = False
theta180_computed = False

for theta, phi in product(thetas, phis):
    if theta == 0 and theta0_computed:
        continue
    elif theta == 180 and theta180_computed:
        continue
    plt.close(1)
    fig = plt.figure(num=1, figsize=(16, 7), dpi=72)
    ax1 = fig.add_subplot(121, projection='3d')
    ax2 = fig.add_subplot(122)

    ax_theta = np.array([np.deg2rad(theta)], dtype=ftype)
    ax_phi = np.array([np.deg2rad(phi)], dtype=ftype)
    sph2cart(ax_r, ax_theta, ax_phi, ax_x, ax_y, ax_z)

    orig_ax = [0, 0, 1]
    new_ax = [float(ax_x), float(ax_y), float(ax_z)]

    ct = np.cos(float(ax_theta))
    st = np.sin(float(ax_theta))
    Ry = np.array(
        [[ct, 0, st],
         [0, 1, 0],
         [-st, 0, ct]]
    )
    cp = np.cos(float(ax_phi))
    sp = np.sin(float(ax_phi))
    Rz = np.array(
        [[cp, -sp, 0],
         [sp, cp, 0],
         [0, 0, 1]]
    )
    new_points = np.matmul(Rz, np.matmul(Ry, points))
    fig, ax, pts, sph = plot_points_on_sph(
        new_points[0,:], new_points[1,:], new_points[2,:],
        ax_x, ax_y, ax_z, ax=ax1
    )

    # Convert to spherical coords

    new_r, new_theta, new_phi = np.empty(n_samp, dtype=ftype), np.empty(n_samp, dtype=ftype), np.empty(n_samp, dtype=ftype)
    cart2sph(new_points[0,:], new_points[1,:], new_points[2,:], new_r, new_theta, new_phi)
    new_costheta = np.cos(new_theta)
    assert np.allclose(new_r, 1)
    dpdir_b = OneDimBinning('deltaphidir', tex=r'\Delta\phi_{\rm dir}',
                            domain=[0, np.pi], is_lin=True, num_bins=40)
    ctdir_b = OneDimBinning('costhetadir', tex=r'\cos\theta_{\rm dir}',
                            domain=[-1, 1], is_lin=True, num_bins=80)
    binning = dpdir_b * ctdir_b
    hist, edges = np.histogramdd(
        (np.abs(new_phi - bin_phi), new_costheta),
        bins=binning.bin_edges,
        normed=False
    )
    hist = hist / hist.sum()
    hist = Map(
        name='hist',
        tex=(r'{\rm Bin \; weights}, \; \theta=%.1f^{\circ} \; {\rm and} \; \phi=%.1f^{\circ}'
             % (np.rad2deg(theta), np.rad2deg(phi))),
        hist=hist,
        binning=binning
    )
    #print('number of nonzero bins:', np.sum(hist.hist > 0))
    hist.plot(vmax=0.03, ax=ax2, titlesize=18, xlabelsize=16, ylabelsize=16)
    fig.tight_layout()
    fig.savefig(
        join(outdir, 'theta{}_phi{}.png'.format(theta, phi)),
        dpi=120
    )
    if theta == 0:
        theta0_computed = True
    elif theta == 180:
        theta180_computed = True
pass
```

```python
import sympy
from sympy import (
    acos, asin, sin, cos, tan, atan, atan2, sqrt, symbols, Matrix,
    simplify, init_printing, trigsimp
)
init_printing()
```

```python
theta, phi = symbols('theta, phi')
p_x, p_y, p_z = symbols('p_x, p_y, p_z')
p_r, p_theta, p_phi = symbols('p_r p_theta p_phi')
p_r = 1
```

```python
p_z = cos(p_theta)
p_rho = sin(p_theta)
p_x = p_rho * cos(p_phi)
p_y = p_rho * sin(p_phi)
```

Create the rotation matrix

```python
# Rotation about y axis
Ry = sympy.Matrix(
    [[ cos(theta),  0, sin(theta)],
     [          0,  1,          0],
     [-sin(theta),  0, cos(theta)]]
)

# Rotation about z axis
Rz = sympy.Matrix(
    [[cos(phi), -sin(phi),  0],
     [sin(phi),  cos(phi),  0],
     [       0,         0,  1]]
)

# Total rotation is first rotate about y axis and then rotate about z axis
R = trigsimp(Rz * Ry)
R
```

Apply the rotation matrix to the pdir vector with components $p_x$, $p_y$, and $p_z$.
The rotated vector is $p'$ (p_prime).

```python
p_prime = simplify(trigsimp(R * Matrix([[p_x], [p_y], [p_z]])))
p_prime
```

```python
costheta_prime = p_prime[2]
costheta_prime
```

```python
phi_prime = simplify(trigsimp(atan2(p_prime[1], p_prime[0])))
phi_prime
```

Translate the above symbolic bits into a numba-fied function.

```python
@numba_jit
def rotate_circle_on_sphere(p_theta, p_phi, rot_theta, rot_phi):
    """
    Parameters
    ----------
    p_theta : scalar float
        All points in the circle share the same theta coordinate.

    p_phi : 1D array
        Azimuth of each point on the circle

    rot_theta : scalar float
        Rotate the circle to have axis of symmetry defined by (rot_theta, rot_phi)

    rot_phi : scalar float
        Rotate the circle to have axis of symmetry defined by (rot_theta, rot_phi)

    Returns
    -------
    q_costheta : 1d array, same size as `p_phi`
        The points on the rotated circle have these costheta coordinates.

    q_phi : 1d array, same size as `p_phi`
        The points on the rotated circle have these phi coordinates.

    """
    num_p = len(p_phi)
    q_costheta = np.empty(num_p)
    q_phi = np.empty(num_p)

    sin_rot_theta = np.sin(rot_theta)
    cos_rot_theta = np.cos(rot_theta)

    sin_rot_phi = np.sin(rot_phi)
    cos_rot_phi = np.cos(rot_phi)

    sin_p_theta = np.sin(p_theta)
    cos_p_theta = np.cos(p_theta)

    for idx in range(num_p):
        p_phi_ = p_phi[idx]
        sin_p_phi = np.sin(p_phi_)
        cos_p_phi = np.cos(p_phi_)
        q_costheta[idx] = -sin_p_theta * sin_rot_theta * cos_p_phi + cos_p_theta * cos_rot_theta
        q_phi[idx] = np.arctan2(
            (sin_p_phi * sin_p_theta * cos_rot_phi) + (sin_p_theta * sin_rot_phi * cos_p_phi * cos_rot_theta) + (sin_rot_phi * sin_rot_theta * cos_p_theta),
            (-sin_p_phi * sin_p_theta * sin_rot_phi) + (sin_p_theta * cos_p_phi * cos_rot_phi * cos_rot_theta) + (sin_rot_theta * cos_p_theta * cos_rot_phi)
        )

    return q_costheta, q_phi
```

Test the rotation code with visualization.

```python
%%time

p_theta = cherenkov_angle
p_phi = np.linspace(0, 2*np.pi*(1 - 1/n_samp), n_samp)

thetas = np.arange(0, 180+1, 15)
phis = np.arange(0, 360, 15)

dpdir_b = OneDimBinning('deltaphidir', tex=r'\Delta\phi_{\rm dir}',
                        domain=[0, np.pi], is_lin=True, num_bins=40)
ctdir_b = OneDimBinning('costhetadir', tex=r'\cos\theta_{\rm dir}',
                        domain=[-1, 1], is_lin=True, num_bins=80)
binning = dpdir_b * ctdir_b
bin_edges = tuple(be.m for be in binning.bin_edges)

ax_x, ax_y, ax_z = np.empty(1, dtype=ftype), np.empty(1, dtype=ftype), np.empty(1, dtype=ftype)

theta0_computed = False
theta180_computed = False

for iter_num, (theta, phi) in enumerate(product(thetas, phis)):
    #if iter_num != 50:
    #    continue
    if theta == 0 and theta0_computed:
        continue
    elif theta == 180 and theta180_computed:
        continue

    plt.close(1)
    fig = plt.figure(num=1, figsize=(16, 7), dpi=72)
    ax1 = fig.add_subplot(121, projection='3d')
    ax2 = fig.add_subplot(122)

    rot_theta = np.deg2rad(theta)
    rot_phi = np.deg2rad(phi)

    sph2cart(np.array([1.0]), np.array([rot_theta]), np.array([rot_phi]), ax_x, ax_y, ax_z)

    q_costheta, q_phi = rotate_circle_on_sphere(p_theta, p_phi, rot_theta, rot_phi)
    q_z = q_costheta
    q_rho = np.sqrt(1 - q_z**2)
    q_x = q_rho * np.cos(q_phi)
    q_y = q_rho * np.sin(q_phi)

    fig, ax, pts, sph = plot_points_on_sph(
        q_x, q_y, q_z,
        ax_x, ax_y, ax_z, ax=ax1
    )

    hist, edges = np.histogramdd(
        (np.abs(q_phi - bin_phi), q_costheta),
        bins=bin_edges,
        normed=False
    )
    hist = hist / hist.sum()
    hist = Map(
        name='hist',
        tex=(r'{\rm Bin \; weights}, \; \theta=%.0f^{\circ} \; {\rm and} \; \phi=%.0f^{\circ}'
             % (np.rad2deg(rot_theta), np.rad2deg(rot_phi))),
        hist=hist,
        binning=binning
    )
    #print('number of nonzero bins:', np.sum(hist.hist > 0))
    hist.plot(vmax=0.03, ax=ax2, titlesize=18, xlabelsize=16, ylabelsize=16)
    fig.tight_layout()
    fig.savefig(
        join(outdir, 'theta{}_phi{}.png'.format(theta, phi)),
        dpi=120
    )
    if theta == 0:
        theta0_computed = True
    elif theta == 180:
        theta180_computed = True
    sys.stdout.write('{} '.format(iter_num))
    sys.stdout.flush()

sys.stdout.write('\n')
pass
```

Let's define a single function that can rotate the points, bin them, and return bin indices and weights.

```python
@numba_jit
def test_idx():
    out = [(0,0)]
    counts = [1]
    for i in range(1000):
        if (0,0) not in out:
            out.append((0,0))
            counts.append(1)

        if (0,1) in out:
            idx = out.index((0,1))
            counts[idx] += 1

        else:
            out.append((0,1))
            counts.append(1)

    return out, counts

# Run once to compile the func
out, counts = test_idx()
```

```python
%timeit out, counts = test_idx()
```

    10000 loops, best of 3: 71 µs per loop

```python
@numba_jit
def cone_weights(theta, num_phi, rot_theta, rot_phi, num_costheta_bins, num_phi_bins):
    """
    Parameters
    ----------
    theta : scalar float
        The Cherenkov angle (i.e., half the cone's opening angle)

    num_phi : scalar int
        Number of azimuth samples of the circle where the cone intersects the unit sphere

    rot_theta : scalar float
        Rotate the cone to have axis of symmetry defined by (rot_theta, rot_phi)

    rot_phi : scalar float
        Rotate the cone to have axis of symmetry defined by (rot_theta, rot_phi)

    num_costheta_bins, num_phi_bins : scalar int

    Returns
    -------
    bin_indices : list of 2-tuples
        Indices of (costheta_dir, deltaphi_dir) bins which carry non-zero weights

    weights : 1D numpy.ndarray

    """
    sin_rot_theta = np.sin(rot_theta)
    cos_rot_theta = np.cos(rot_theta)

    sin_rot_phi = np.sin(rot_phi)
    cos_rot_phi = np.cos(rot_phi)

    sin_theta = np.sin(theta)
    cos_theta = np.cos(theta)

    costheta_bin_width = 2.0 / num_costheta_bins
    phi_bin_width = np.pi / num_phi_bins

    bin_indices = []
    counts = []
    counts_total = 0.0

    for idx in range(num_phi):
        p_phi = 2*np.pi * float(idx)/float(num_phi + 1)
        sin_p_phi = np.sin(p_phi)
        cos_p_phi = np.cos(p_phi)
        q_costheta = (-sin_theta * sin_rot_theta * cos_p_phi) + (cos_theta * cos_rot_theta)
        abs_q_phi = np.abs(np.arctan2(
            (sin_p_phi * sin_theta * cos_rot_phi) + (sin_theta * sin_rot_phi * cos_p_phi * cos_rot_theta) + (sin_rot_phi * sin_rot_theta * cos_theta),
            (-sin_p_phi * sin_theta * sin_rot_phi) + (sin_theta * cos_p_phi * cos_rot_phi * cos_rot_theta) + (sin_rot_theta * cos_theta * cos_rot_phi)
        ))
        costheta_bin = int((q_costheta + 1) // costheta_bin_width)
        phi_bin = int(abs_q_phi // phi_bin_width)
        coord = (costheta_bin, phi_bin)
        if coord in bin_indices:
            idx = bin_indices.index(coord)
            counts[idx] += 1.0
        else:
            bin_indices.append(coord)
            counts.append(1.0)
        counts_total += 1.0

    num_nonzero_bins = len(counts)

    weights = np.empty(num_nonzero_bins)
    for idx in range(num_nonzero_bins):
        weights[idx] = counts[idx] / counts_total

    return bin_indices, weights

# Run once to compile the function
bin_indices, weights = cone_weights(theta=cherenkov_angle, num_phi=1000, rot_theta=0.0, rot_phi=0.0, num_costheta_bins=80, num_phi_bins=40)
```

```python
rot_theta = np.deg2rad(30)
rot_phi = np.deg2rad(45)

%timeit bin_indices, weights = cone_weights(theta=cherenkov_angle, num_phi=10000, rot_theta=rot_theta, rot_phi=rot_phi, num_costheta_bins=80, num_phi_bins=40)
```

    100 loops, best of 3: 2.73 ms per loop

```python
rot_theta = np.deg2rad(30)
rot_phi = np.deg2rad(45)
num_costheta_bins = 80
num_phi_bins = 40

bin_indices, weights = cone_weights(
    theta=cherenkov_angle,
    num_phi=10000,
    rot_theta=rot_theta,
    rot_phi=rot_phi,
    num_costheta_bins=num_costheta_bins,
    num_phi_bins=num_phi_bins
)
bin_indices = np.array(bin_indices)
hist = np.zeros((num_costheta_bins, num_phi_bins))
hist[bin_indices[:,0], bin_indices[:,1]] = weights

plt.close(1)
fig = plt.figure(num=1, figsize=(8, 7), dpi=72)
ax2 = fig.add_subplot(111)

hist = Map(
    name='hist',
    tex=(r'{\rm Bin \; weights}, \; \theta=%.0f^{\circ} \; {\rm and} \; \phi=%.0f^{\circ}'
         % (np.rad2deg(rot_theta), np.rad2deg(rot_phi))),
    hist=hist.T,
    binning=binning
)
hist.plot(vmax=0.03, ax=ax2, titlesize=18, xlabelsize=16, ylabelsize=16)
fig.tight_layout()
pass
```

Let's take some timings to see how long generating a mask and weights will take as a function of `n_samp`.

```python
rot_theta = np.deg2rad(30)
rot_phi = np.deg2rad(30)
num_trials = 100

results = []
for num_phi in [int(1e2), int(1e3), int(1e4), int(1e5)]:
    result = dict(num_phi=num_phi)
    p_phi = np.linspace(0, 2*np.pi*(1 - 1/n_samp), n_samp)
    times = []
    for trial in range(num_trials):
        step_fract = trial / num_trials
        t0 = time()
        bin_indices, weights = cone_weights(
            theta=cherenkov_angle*step_fract,
            num_phi=num_phi,
            rot_theta=rot_theta*step_fract,
            rot_phi=rot_phi*step_fract,
            num_costheta_bins=80,
            num_phi_bins=40
        )
        t1 = time()
        times.append(t1 - t0)
    result['time_ms'] = np.round((np.mean(times)) * 1e3, 3)
    results.append(result)

results = pd.DataFrame(results)
results
```

      num_phi
      time_ms

      0
      100
      0.053

      1
      1000
      0.404

      2
      10000
      2.970

      3
      100000
      23.145

Not that good, but we can revisit optimizing this later (if needed).

Okay, the last step is that we need to actually apply this to a table, so let's modify the function to directly sample from a 5D table (as produced by CLSim).

```python
@numba_jit
def survival_prob_from_cone(theta, num_phi, rot_theta, rot_phi, directional_survival_prob):
    """
    Parameters
    ----------
    theta : scalar float
        Cherenkov angle (half the cone's opening angle)

    num_phi : scalar int
        Number of azimuth samples of the circle where the cone intersects the unit sphere

    rot_theta : scalar float
        Rotate the cone to have axis of symmetry defined by (rot_theta, rot_phi)

    rot_phi : scalar float
        Rotate the cone to have axis of symmetry defined by (rot_theta, rot_phi)

    directional_survival_prob : ndarray of shape (N_costhetadir x N_deltaphidir)

    Returns
    -------
    survival_prob : scalar float

    """
    num_costheta_bins, num_phi_bins = directional_survival_prob.shape

    sin_rot_theta = np.sin(rot_theta)
    cos_rot_theta = np.cos(rot_theta)

    sin_rot_phi = np.sin(rot_phi)
    cos_rot_phi = np.cos(rot_phi)

    sin_theta = np.sin(theta)
    cos_theta = np.cos(theta)

    costheta_bin_width = 2.0 / num_costheta_bins
    deltaphi_bin_width = np.pi / num_phi_bins

    bin_indices = []
    counts = []
    counts_total = 0.0

    for idx in range(num_phi):
        p_phi = 2*np.pi * float(idx)/float(num_phi + 1)
        sin_p_phi = np.sin(p_phi)
        cos_p_phi = np.cos(p_phi)
        q_costheta = (-sin_theta * sin_rot_theta * cos_p_phi) + (cos_theta * cos_rot_theta)
        abs_q_phi = np.abs(np.arctan2(
            (sin_p_phi * sin_theta * cos_rot_phi) + (sin_theta * sin_rot_phi * cos_p_phi * cos_rot_theta) + (sin_rot_phi * sin_rot_theta * cos_theta),
            (-sin_p_phi * sin_theta * sin_rot_phi) + (sin_theta * cos_p_phi * cos_rot_phi * cos_rot_theta) + (sin_rot_theta * cos_theta * cos_rot_phi)
        ))
        costheta_bin = int((q_costheta + 1) // costheta_bin_width)
        deltaphi_bin = int(abs_q_phi // deltaphi_bin_width)
        coord = (costheta_bin, deltaphi_bin)
        if coord in bin_indices:
            idx = bin_indices.index(coord)
            counts[idx] += 1
        else:
            bin_indices.append(coord)
            counts.append(1)
        counts_total += 1

    num_nonzero_bins = len(counts)
    survival_prob = 0.0
    for idx, count in zip(bin_indices, counts):
        survival_prob += directional_survival_prob[idx] * count

    survival_prob /= float(counts_total)

    return survival_prob

# Run once to compile the function
survival_prob = survival_prob_from_cone(theta=cherenkov_angle, num_phi=1000, rot_theta=0.0, rot_phi=0.0, directional_survival_prob=np.ones((10, 20)))
```

## Test the code as it was ultimately implemented in Retro

```python
ctdir_b = OneDimBinning(
    'costhetadir', tex=r'\cos\theta_{\rm dir}',
    domain=[-1, 1], is_lin=True, num_bins=80
)
dpdir_b = OneDimBinning(
    'deltaphidir', tex=r'\Delta\phi_{\rm dir}',
    domain=[0, np.pi], is_lin=True, num_bins=40
)
dir_binning = ctdir_b * dpdir_b
survival_prob = dir_binning.ones(name='survival_prob')
```

```python
COS_CKV = 0.764540803152 ; THETA_CKV = np.arccos(COS_CKV)
rot_theta = 180-np.rad2deg(THETA_CKV) # deg
rot_phi = 180 # deg
num_phi = int(1e5)
sigma_theta = 10 # deg

common_kwargs = dict(
    num_phi=num_phi,
    rot_costheta=np.cos(np.deg2rad(rot_theta)),
    rot_sintheta=np.sin(np.deg2rad(rot_theta)),
    rot_cosphi=np.cos(np.deg2rad(rot_phi)),
    rot_sinphi=np.sin(np.deg2rad(rot_phi)),
    directional_survival_prob=survival_prob.hist,
    num_costheta_bins=ctdir_b.num_bins,
    num_deltaphi_bins=dpdir_b.num_bins,
)
```

First, test the "simple" code that does not smear the points on the Cherenkov cone.

```python
smsp, bin_indices, counts = survival_prob_from_cone(
    costheta=COS_CKV,
    sintheta=np.sin(THETA_CKV),
    **common_kwargs
)
total_counts = np.sum(counts)
weight_map = dir_binning.zeros(name='Weight map')
for bin_idx, count in zip(bin_indices, counts):
    weight_map.hist[bin_idx] = count / total_counts
transp = weight_map.reorder_dimensions(('deltaphidir', 'costhetadir'))
transp.plot(fig_kw=dict(figsize=(8, 6), dpi=120))
pass
```

Now test the code that smears out the Cherenkov cone.

```python
rand = np.random.RandomState(0)
random_delta_thetas = rand.normal(loc=0, scale=np.deg2rad(sigma_theta), size=num_phi)
smsp, bin_indices, counts = survival_prob_from_smeared_cone(
   theta=THETA_CKV,
   random_delta_thetas=random_delta_thetas,
   **common_kwargs
)
total_counts = np.sum(counts)
weight_map = dir_binning.zeros(name='Weight map')
for bin_idx, count in zip(bin_indices, counts):
    weight_map.hist[bin_idx] = count / total_counts
transp = weight_map.reorder_dimensions(('deltaphidir', 'costhetadir'))
transp.plot(fig_kw=dict(figsize=(8, 6), dpi=120))
pass
```


| |Pierre Proulx, ing, professeur|
|:---|:---|
|Département de génie chimique et de génie biotechnologique |** GCH200-Phénomènes d'échanges I **|

### Section 10.6, Conduction de la chaleur dans un plan

```python
#
# Pierre Proulx
#
# Préparation de l'affichage et des outils de calcul symbolique
#
import sympy as sp
from IPython.display import *
sp.init_printing(use_latex=True)
%matplotlib inline
```

```python
# Paramètres, variables et fonctions
x,k01,k12,k23,h0,h3=sp.symbols('x k_1 k_2 k_3 h_0 h_3')
x0,x1,x2,x3,Ta,Tb=sp.symbols('x_0 x_1 x_2 x_3 T_a T_b')
q=sp.symbols('q')
T=sp.Function('T')(x)
```

```python
eq1=sp.Eq(k01*sp.Derivative(sp.Derivative(T,x)),0)
eq2=sp.Eq(k12*sp.Derivative(sp.Derivative(T,x)),0)
eq3=sp.Eq(k23*sp.Derivative(sp.Derivative(T,x)),0)
T1=sp.dsolve(eq1).rhs
T2=sp.dsolve(eq2)
T2=T2.subs(sp.symbols('C1'),sp.symbols('C3'))
T2=T2.subs(sp.symbols('C2'),sp.symbols('C4')).rhs
T3=sp.dsolve(eq3)
T3=T3.subs(sp.symbols('C1'),sp.symbols('C5'))
T3=T3.subs(sp.symbols('C2'),sp.symbols('C6')).rhs
display(T1)
display(T2)
display(T3)
```

```python
# Maintenant on pose les conditions aux limites pour trouver les 6 constantes
cl1=sp.Eq(T1.subs(x,x1)-T2.subs(x,x1))   # températures égales sur les points intérieurs
cl2=sp.Eq(T2.subs(x,x2)-T3.subs(x,x2))
#                           flux égaux sur les points intérieurs
cl3=sp.Eq(k01*T1.diff(x).subs(x,x1)-k12*T2.diff(x).subs(x,x1))
cl4=sp.Eq(k12*T2.diff(x).subs(x,x2)-k23*T3.diff(x).subs(x,x2))
#                           flux donnés par la loi de refroidissement de Newton sur les parois
cl5=sp.Eq(-k01*T1.diff(x).subs(x,x0)+h0*(T1.subs(x,x0)-Ta))
cl6=sp.Eq(-k23*T3.diff(x).subs(x,x3)+h3*(Tb-T3.subs(x,x3)))
```

```python
constantes=sp.solve((cl1,cl2,cl3,cl4,cl5,cl6),sp.symbols('C1 C2 C3 C4 C5 C6'))
T1=T1.subs(constantes)
T2=T2.subs(constantes)
T3=T3.subs(constantes)
display('profils de températures dans les trois plans avant de substituer des valeurs',T1,T2,T3)
dico={'k_1':4,'k_2':25,'k_3':1,
      'h_0':1000,'h_3':2000,'x_0':0.0,'x_1':0.025,'x_2':0.050,'x_3':0.085,'T_a':100,'T_b':20}
T1p=T1.subs(dico).evalf(3)
T2p=T2.subs(dico).evalf(3)
T3p=T3.subs(dico).evalf(3)
display('profils de températures dans les trois plans après substitution',T1p,T2p,T3p)
```

```python
# Calcule les taux de chaleur en 0 et en 3 (doivent être égaux) (watts)
#
W,L=sp.symbols('W L')
taux3=(h3*(T3-Tb)).subs(dico)*W*L     # pour mettre les valeurs numériques dans
taux0=(h0*(Ta-T1)).subs(dico)*W*L      # l'expression symbolique, on subs(dico)
#
print(taux3.subs(x,x3.subs(dico)), taux0.subs(x,x0.subs(dico)))
import matplotlib.pyplot as plt
plt.rcParams['figure.figsize'] = 12, 10
#plt.figure(figsize=(12,10))
Tap=sp.symbols('Tap')
Tap=Ta
Tbp=sp.symbols('Tbp')
Tbp=Tb
p=sp.plot((Tap.subs(dico),(x,x0.subs(dico),x0.subs(dico)+0.001))
         ,(Tbp.subs(dico),(x,x3.subs(dico)-0.001,x3.subs(dico)))
         ,(T1p,(x,x0.subs(dico),x1.subs(dico)))
         ,(T2p,(x,x1.subs(dico),x2.subs(dico)))
         ,(T3p,(x,x2.subs(dico),x3.subs(dico)))
         ,legend=True,ylabel='T(x)',xlabel='x',show=False) #affiche pas tout de suite
p[0].line_color = 'red'
p[0].label='de x= x_0 à x=x_1 '
p[1].line_color = 'black'
p[1].label='de x = x_1 à x=x_2 '
p[2].line_color = 'green'
p[2].label='de x = x_2 à x=x_3 '
p.show()                         # maintenant on est prêts à afficher
## attention, les deux petites lignes ajoutées en Ta et Tb sont pour montrer la différence
## entre les températures du milieu ambiant et la surface, jouez avec h0 et h3 pour voir...
```