# Semi-Monocoque Theory: corrective solutions

```python
from pint import UnitRegistry
import sympy
import networkx as nx
#import numpy as np
import matplotlib.pyplot as plt
#import sys
%matplotlib inline
from IPython.display import display
```

Import **Section** class, which contains all calculations

```python
from Section import Section
```

Initialization of **sympy** symbolic tool and **pint** for dimension analysis (not really implemented rn as not directly compatible with sympy)

```python
ureg = UnitRegistry()
sympy.init_printing()
```

Define **sympy** parameters used for geometric description of sections

```python
A, A0, t, t0, a, b, h, L, E, G = sympy.symbols('A A_0 t t_0 a b h L E G', positive=True)
```

We also define numerical values for each **symbol** in order to plot scaled section and perform calculations

```python
values = [(A, 150 * ureg.millimeter**2),(A0, 250  * ureg.millimeter**2),(a, 80 * ureg.millimeter), \
          (b, 20 * ureg.millimeter),(h, 35 * ureg.millimeter),(L, 2000 * ureg.millimeter), \
          (t, 0.8 *ureg.millimeter),(E, 72e3 * ureg.MPa), (G, 27e3 * ureg.MPa)]
datav = [(v[0],v[1].magnitude) for v in values]
```

# Third example: Simple rectangular section with 7 nodes Non symmetric

Define graph describing the section:

1) **stringers** are **nodes** with parameters:
- **x** coordinate
- **y** coordinate
- **Area**

2) **panels** are **oriented edges** with parameters:
- **thickness**
- **lenght** which is automatically calculated

```python
stringers = {1:[(3*a,h),A],
             2:[(2*a,h),A],
             3:[(a,h),A],
             4:[(sympy.Integer(0),h),A],
             5:[(sympy.Integer(0),sympy.Integer(0)),A],
             6:[(sympy.Rational(7,4)*a,sympy.Integer(0)),A],
             7:[(3*a,sympy.Integer(0)),A]}

panels = {(1,2):t,
          (2,3):t,
          (3,4):t,
          (4,5):t,
          (5,6):t,
          (6,7):t,
          (7,1):t}
```

Define section and perform first calculations

```python
S1 = Section(stringers, panels)
```

```python
S1.cycles
```

## Plot of **S1** section in original reference frame

Define a dictionary of coordinates used by **Networkx** to plot section as a Directed graph.
Note that arrows are actually just thicker stubs

```python
start_pos={ii: [float(S1.g.node[ii]['ip'][i].subs(datav)) for i in range(2)] for ii in S1.g.nodes() }
```

```python
plt.figure(figsize=(12,8),dpi=300)
nx.draw(S1.g,with_labels=True, arrows= True, pos=start_pos)
plt.arrow(0,0,20,0)
plt.arrow(0,0,0,20)
#plt.text(0,0, 'CG', fontsize=24)
plt.axis('equal')
plt.title("Section in starting reference Frame",fontsize=16);
```

## Plot of **S1** section in inertial reference Frame

Section is plotted wrt **center of gravity** and rotated (if necessary) so that *x* and *y* are principal axes.
**Center of Gravity** and **Shear Center** are drawn

```python
positions={ii: [float(S1.g.node[ii]['pos'][i].subs(datav)) for i in range(2)] for ii in S1.g.nodes() }
```

```python
x_ct, y_ct = S1.ct.subs(datav)

plt.figure(figsize=(12,8),dpi=300)
nx.draw(S1.g,with_labels=True, pos=positions)
plt.plot([0],[0],'o',ms=12,label='CG')
plt.plot([x_ct],[y_ct],'^',ms=12, label='SC')
#plt.text(0,0, 'CG', fontsize=24)
#plt.text(x_ct,y_ct, 'SC', fontsize=24)
plt.legend(loc='lower right', shadow=True)
plt.axis('equal')
plt.title("Section in pricipal reference Frame",fontsize=16);
```

Expression of **inertial properties** in *principal reference frame*

```python
sympy.simplify(S1.Ixx), sympy.simplify(S1.Iyy), sympy.simplify(S1.Ixy), sympy.simplify(S1.θ)
```

```python
S1.symmetry
```

    [{'edges': [], 'nodes': []}, {'edges': [], 'nodes': []}]

```python
S1.compute_L()
```

```python
S1.L
```

```python
S1.compute_H()
```

```python
S1.H.subs(datav)
```

```python
S1.compute_KM(A,h,t)
```

```python
S1.Ktilde
```

```python
S1.Mtilde.subs(datav)
```

```python
sol_data = (S1.Ktilde.inv()*(S1.Mtilde.subs(datav))).eigenvects()
sol_data
```

```python
β2 = [sol[0] for sol in sol_data]
β2
```

```python
X = []
for sol in sol_data:
    for i in range(len(sol[2])):
        X.append(sympy.N(sol[2][i]/sol[2][i].norm()))
X
```

```python
λ = [sympy.N(sympy.sqrt(E*A*h/(G*t)*βi).subs(datav)) for βi in β2]
λ
```

```python

```


Simulación matemática 2018

Lázaro Alonso

Email:   `alonsosilva@iteso.mx, lazarus.alon@gmail.com`

< [Optimización de funciones SymPy](Clase4_OptimizacionSympy.ipynb) | [Guía](Clase0_GuiaSimulacionM.ipynb) | [Ajuste de curvas](Clase6_AjusteCurvas.ipynb) >
___

### Programación lineal

> La programación lineal es el campo de la optimización matemática dedicado a maximizar o minimizar (optimizar) funciones lineales, denominada función objetivo, de tal forma que las variables de dicha función estén sujetas a una serie de restricciones expresadas mediante un sistema de ecuaciones o inecuaciones también lineales.

**Referencias:**
- https://es.wikipedia.org/wiki/Programaci%C3%B3n_lineal
- https://docs.scipy.org/doc/scipy-0.18.1/reference/optimize.html

## 1. Apuntes históricos

- 1826:	Joseph Fourier anticipa la programación lineal. Carl Friedrich Gauss resuelve ecuaciones lineales por eliminación "gaussiana".
- 1902:	Gyula Farkas concibe un método para resolver sistemas de inecuaciones.
- Es hasta la Segunda Guerra Mundial que se plantea la programación lineal como un modelo matemático para planificar gastos y retornos, de modo que se reduzcan costos de guerra y aumentar pérdidas del enemigo. Secreto hasta 1947 (posguerra).
- 1947:	George Dantzig publica el algoritmo simplex y John von Neumann desarrolló la teoría de la dualidad. Se sabe que Leonid Kantoróvich también formuló la teoría en forma independiente.
- Fue usado por muchas industrias en la planificación diaria.

**Hasta acá, tiempos exponenciales de solución. Lo siguiente, tiempo polinomial.**

- 1979: Leonid Khachiyan, diseñó el llamado Algoritmo del elipsoide, a través del cual demostró que el problema de la programación lineal es resoluble de manera eficiente, es decir, en tiempo polinomial.
- 1984: Narendra Karmarkar introduce el método del punto interior para resolver problemas de programación lineal.

**Mencionar complejidad computacional.**

## 2. Motivación

Ya la clase pasada habíamos mencionado que cuando se quería optimizar una función de varias variables con restricciones, se podía aplicar siempre el método de Multiplicadores de Lagrange. Sin embargo, este método es computacionalmente muy complejo conforme crece el número de variables.

Por tanto, cuando la función a optimizar y las restricciones son de caracter lineal, los métodos de solución que se pueden desarrollar son computacionalmente eficientes, por lo que es útil realizar la distinción.

## 3. Problemas de programación lineal

De acuerdo a lo descrito anteriormente, un problema de programación lineal puede escribirse en la siguiente forma:

\begin{equation}
\begin{array}{ll}
\min_{x_1,\dots,x_n} & c_1x_1+\dots+c_nx_n \\
\text{s. a. }        & a^{eq}_{j,1}x_1+\dots+a^{eq}_{j,n}x_n=b^{eq}_j \text{ para } 1\leq j\leq m_1 \\
                     & a_{k,1}x_1+\dots+a_{k,n}x_n\leq b_k \text{ para } 1\leq k\leq m_2,
\end{array}
\end{equation}
donde:
- $x_i$ para $i=1,\dots,n$ son las incógnitas o variables de decisión,
- $c_i$ para $i=1,\dots,n$ son los coeficientes de la función a optimizar,
- $a^{eq}_{j,i}$ para $j=1,\dots,m_1$ e $i=1,\dots,n$, son los coeficientes de la restricción de igualdad,
- $a_{k,i}$ para $k=1,\dots,m_2$ e $i=1,\dots,n$, son los coeficientes de la restricción de desigualdad,
- $b^{eq}_j$ para $j=1,\dots,m_1$ son valores conocidos que deben ser respetados estrictamente, y
- $b_k$ para $k=1,\dots,m_2$ son valores conocidos que no deben ser superados.

Equivalentemente, el problema puede escribirse como

\begin{equation}
\begin{array}{ll}
\min_{\boldsymbol{x}} & \boldsymbol{c}^T\boldsymbol{x} \\
\text{s. a. }        & \boldsymbol{A}_{eq}\boldsymbol{x}=\boldsymbol{b}_{eq} \\
                     & \boldsymbol{A}\boldsymbol{x}\leq\boldsymbol{b},
\end{array}
\end{equation}
donde:
- $\boldsymbol{x}=\left[x_1\quad\dots\quad x_n\right]^T$,
- $\boldsymbol{c}=\left[c_1\quad\dots\quad c_n\right]^T$,
- $\boldsymbol{A}_{eq}=\left[\begin{array}{ccc}a^{eq}_{1,1} & \dots & a^{eq}_{1,n}\\ \vdots & \ddots & \vdots\\ a^{eq}_{m_1,1} & \dots & a^{eq}_{m_1,n}\end{array}\right]$,
- $\boldsymbol{A}=\left[\begin{array}{ccc}a_{1,1} & \dots & a_{1,n}\\ \vdots & \ddots & \vdots\\ a_{m_2,1} & \dots & a_{m_2,n}\end{array}\right]$,
- $\boldsymbol{b}_{eq}=\left[b^{eq}_1\quad\dots\quad b^{eq}_{m_1}\right]^T$, y
- $\boldsymbol{b}=\left[b_1\quad\dots\quad b_{m_2}\right]^T$.

**Nota:** el problema $\max_{\boldsymbol{x}}\boldsymbol{g}(\boldsymbol{x})$ es equivalente a $\min_{\boldsymbol{x}}-\boldsymbol{g}(\boldsymbol{x})$.

## 4. Ejemplo básico
Una compañía produce dos productos ($X_1$ y $X_2$) usando dos máquinas ($A$ y $B$). Cada unidad de **$X_1$ que se produce requiere 50 minutos en la máquina $A$** y 30 minutos en la máquina $B$. Cada unidad de **$X_2$ que se produce requiere 24 minutos en la máquina $A$** y 33 minutos en la máquina $B$.

Al comienzo de la semana hay 30 unidades de $X_1$ y 90 unidades de $X_2$ en inventario. El tiempo de uso disponible de la **máquina $A$ es de 40 horas** y el de la máquina $B$ es de 35 horas.

La demanda para $X_1$ en la semana actual es de 75 unidades y de $X_2$ es de 95 unidades. La política de la compañía es maximizar la suma combinada de unidades de $X_1$ e $X_2$ en inventario al finalizar la semana.

Formular el problema de decidir cuánto hacer de cada producto en la semana como un problema de programación lineal.

### Solución

Sean:
- $x_1$ la cantidad de unidades de $X_1$ a ser producidas en la semana, y
- $x_2$ la cantidad de unidades de $X_2$ a ser producidas en la semana.

Notar que lo que se quiere es maximizar $x_1+x_2$.

Restricciones:
1. El tiempo de uso disponible de la máquina $A$ es de 40 horas: $50x_1+24x_2\leq 40(60)\Rightarrow 50x_1+24x_2\leq 2400$.
2. El tiempo de uso disponible de la máquina $B$ es de 35 horas: $30x_1+33x_2\leq 35(60)\Rightarrow 30x_1+33x_2\leq 2100$.
3. La demanda para $X_1$ en la semana actual es de 75 unidades: $x_1+30\geq 75\Rightarrow x_1\geq 45\Rightarrow -x_1\leq -45$.
4. La demanda para $X_2$ en la semana actual es de 95 unidades: $x_2+90\geq 95\Rightarrow x_2\geq 5\Rightarrow -x_2\leq -5$.

$ 50x_1+24x_2\leq 2400$

Finalmente, el problema puede ser expresado en la forma explicada como:
\begin{equation}
\begin{array}{ll}
\min_{x_1,x_2} & -x_1-x_2 \\
\text{s. a. }  & 50x_1+24x_2\leq 2400 \\
               & 30x_1+33x_2\leq 2100 \\
               & -x_1\leq -45 \\
               & -x_2\leq -5,
\end{array}
\end{equation}

o, eqivalentemente
\begin{equation}
\begin{array}{ll}
\min_{\boldsymbol{x}} & \boldsymbol{c}^T\boldsymbol{x} \\
\text{s. a. }        & \boldsymbol{A}_{eq}\boldsymbol{x}=\boldsymbol{b}_{eq} \\
                     & \boldsymbol{A}\boldsymbol{x}\leq\boldsymbol{b},
\end{array}
\end{equation}
con
- $\boldsymbol{c}=\left[-1 \quad -1\right]^T$,
- $\boldsymbol{A}=\left[\begin{array}{cc}50 & 24 \\ 30 & 33\\ -1 & 0\\ 0 & -1\end{array}\right]$, y
- $\boldsymbol{b}=\left[2400\quad 2100\quad -45\quad -5\right]^T$.

Preferiremos, en adelante, la notación vectorial/matricial.

Este problema está sencillo pues solo es en dos variables. La solución gráfica es válida.

### Solución gráfica

```python
import matplotlib.pyplot as plt
%matplotlib inline
import numpy as np
```

```python
def x2_v1(x1):
    return (2400 - 50*x1)/24
def x2_v2(x1):
    return (2100 - 30*x1)/33
```

```python
x1 = np.linspace(43, 47)
```

```python
plt.plot(x1, x2_v1(x1), 'magenta') #desigualdad 1
plt.plot(x1, x2_v2(x1)) # desigualdad 2
plt.plot([45, 45], [0, 25], 'k')
plt.plot([43, 47], [5, 5], 'r')
plt.xlim(xmin = 44, xmax = 46)
plt.ylim(ymin = 4.5, ymax = 6.5)
plt.scatter([45], [6.25], color = 'red')
plt.fill_between(np.array([45, 45.6]), x2_v1(np.array([45, 45.6])),
                5*np.ones(2), alpha = .2, color = 'orange')
plt.show()
```

## Usando las librerías de Python

### 5.1 Librería `SciPy`

`SciPy` es un softwar de código abierto basado en `Python` para matemáticas, ciencia e ingeniería.

En particular, los siguientes son algunos de los paquetes básicos:
- `NumPy`
- **Librería `SciPy`**
- `SymPy`
- `matplotlib`
- `pandas`

La **Librería `SciPy`** es uno de los paquetes principales y provee varias rutinas numéricas eficientes. Entre ellas, para integración numérica y optimización.

En esta clase, y en lo que resta del módulo, estaremos utilizando el módulo `optimize` de la librería `SciPy`.

**Importémoslo**

```python
import scipy.optimize as opt
```

El módulo `optimize` que acabamos de importar contiene varias funciones para optimización y búsqueda de raices ($f(x)=0$). Entre ellas se encuentra la función `linprog`

```python
#help(opt.linprog)
```

la cual resuelve problemas como los que aprendimos a plantear.

### 5.2 Solución del ejemplo básico con linprog
Ya hicimos la solución gráfica. Contrastemos con la solución que nos da `linprog`...

```python
import numpy as np
```

```python
c = np.array([-1, -1])
A = np.array([[50, 24], [30, 33], [-1, 0], [0, -1]])
b = np.array([2400, 2100, -45, -5])
```

```python
A, b, c
```

    (array([[50, 24],
            [30, 33],
            [-1,  0],
            [ 0, -1]]), array([2400, 2100,  -45,   -5]), array([-1, -1]))

```python
res = opt.linprog(c, A_ub=A, b_ub=b, options={'disp': True})
```

    Optimization terminated successfully.
             Current function value: -51.250000
             Iterations: 4

```python
res
```

         fun: -51.25
     message: 'Optimization terminated successfully.'
         nit: 4
       slack: array([  0.  , 543.75,   0.  ,   1.25])
      status: 0
     success: True
           x: array([45.  ,  6.25])

```python
res.x
```

    array([45.  ,  6.25])

Las cantidades de $X_1$ y $X_2$ que se deben producir para maximizar el inventario al final de la semana, con las restricciones de tiempo de uso de las máquinas y de demanda es:

$$x_1=45$$

$$x_2=6.25$$

**Otra forma:** poner las cotas de las variables a parte

```python
c = np.array([-1, -1])
A = np.array([[50, 24], [30, 33]])
b = np.array([2400, 2100])
x1_bound = (45, None)
x2_bound = (5, None)
```

```python
res = opt.linprog(c, A_ub=A, b_ub=b, bounds=(x1_bound, x2_bound), options={'disp':True})
```

    Optimization terminated successfully.
             Current function value: -51.250000
             Iterations: 4

```python
res
```

         fun: -51.25
     message: 'Optimization terminated successfully.'
         nit: 4
       slack: array([  0.  , 543.75,   0.  ,   1.25])
      status: 0
     success: True
           x: array([45.  ,  6.25])

**Actividad.** Mónica hace aretes y cadenitas de joyería. Es tan buena, que todo lo que hace lo vende.

Le toma 30 minutos hacer un par de aretes y una hora hacer una cadenita, y como Mónica también es estudihambre, solo dispone de 10 horas a la semana para hacer las joyas. Por otra parte, el material que compra solo le alcanza para hacer 15 unidades (el par de aretes cuenta como unidad) de joyas por semana.

La utilidad que le deja la venta de las joyas es \$15 en cada par de aretes y \$20 en cada cadenita.

¿Cuántos pares de aretes y cuántas cadenitas debería hacer Mónica para maximizar su utilidad?

```python
# Resolver acá
import numpy as np
```

```python
c = np.array([-15, -20])
A = np.array([[0.5, 1], [1, 1]])
b = np.array([10, 15])
x1_cota = (0, None)
x2_cota = (0, None)
```

```python
res_monica = opt.linprog(c=c, A_ub=A, b_ub=b, bounds = (x1_cota,x2_cota))
```

```python
res_monica
```

         fun: -250.0
     message: 'Optimization terminated successfully.'
         nit: 2
       slack: array([0., 0.])
      status: 0
     success: True
           x: array([10.,  5.])

#### Formular el problema en la forma explicada y obtener la solución gráfica (puede ser a mano).

**Diez minutos: **

```python

```

#### Ejemplos (un poco mas elaborados)

 - [Problema de transporte](ProbTransporte.ipynb)
 - [Dieta óptima](DietaOptima.ipynb)
 - [Envio de cerveza](EnviarCerveza.ipynb)
 - [Inversión bonos](InversionBonos.ipynb)

___

< [Optimización de funciones SymPy](Clase4_OptimizacionSympy.ipynb) | [Guía](Clase0_GuiaSimulacionM.ipynb) | [Ajuste de curvas](Clase6_AjusteCurvas.ipynb) >

Created with Jupyter by Lázaro Alonso.
  Copyright:  Public Domain como en [CC](https://creativecommons.org/licenses/by/2.0/) (Exepto donde se indique lo contrario)


# Naive Bayes
In this notebook we will create a simple spam classifier from a scratch using Naive Bayes theorem and then we will compare it with *scikit-learn* based classifier. Although Naive Bayes model requires pretty strong assumption to be met (the features of a dataset must be independent), it performs quite well in real life situations and is easy to build and train.

The first step should be importing the necessary libraries and loading a dataset into the DataFrame.

```python
import os
import string
import numpy as np
import pandas as pd
from sklearn import metrics
import matplotlib.pyplot as plt
from wordcloud import WordCloud, STOPWORDS
from sklearn.model_selection import train_test_split

file_name = os.path.join(os.getcwd(), 'data\spam.csv')
df = pd.read_csv(file_name, engine='python')
df.head()
```

      v1
      v2
      Unnamed: 2
      Unnamed: 3
      Unnamed: 4

      0
      ham
      Go until jurong point, crazy.. Available only ...
      NaN
      NaN
      NaN

      1
      ham
      Ok lar... Joking wif u oni...
      NaN
      NaN
      NaN

      2
      spam
      Free entry in 2 a wkly comp to win FA Cup fina...
      NaN
      NaN
      NaN

      3
      ham
      U dun say so early hor... U c already then say...
      NaN
      NaN
      NaN

      4
      ham
      Nah I don't think he goes to usf, he lives aro...
      NaN
      NaN
      NaN

We should remove three last columns because they don't contain any data. Also it's a good idea to rename the first two columns so they will describe content more accurately.

```python
df = df.drop(columns=['Unnamed: 2', 'Unnamed: 3', 'Unnamed: 4'], axis=1)
df = df.rename(columns={'v1': 'label', 'v2': 'message'})
df.head()
```

      label
      message

      0
      ham
      Go until jurong point, crazy.. Available only ...

      1
      ham
      Ok lar... Joking wif u oni...

      2
      spam
      Free entry in 2 a wkly comp to win FA Cup fina...

      3
      ham
      U dun say so early hor... U c already then say...

      4
      ham
      Nah I don't think he goes to usf, he lives aro...

Now we will provide basic exploratory data analysis to describe and visualize our dataset.

```python
df.describe()
```

      label
      message

      count
      5572
      5572

      unique
      2
      5169

      top
      ham
      Sorry, I'll call later

      freq
      4825
      30

```python
df.groupby('label').describe()
```

      message

      count
      unique
      top
      freq

      label

      ham
      4825
      4516
      Sorry, I'll call later
      30

      spam
      747
      653
      Please call our customer service representativ...
      4

```python
df['label'].value_counts().plot(kind='bar', rot=0)
```

```python
def show_wordcloud(label):
    text = ' '.join(df[df['label'] == label]['message'].tolist())
    stopwords = set(STOPWORDS)
    color = 'Reds' if label == 'spam' else 'Greens'

    wordcloud = WordCloud(stopwords=stopwords, background_color='white', colormap=color).generate(text)

    plt.figure(figsize=[10, 8])
    plt.imshow(wordcloud, interpolation='bilinear')
    plt.axis('off')
    plt.show()
```

```python
show_wordcloud('spam')
```

Spam messages tend to have the most frequent words associated with advertisements, sales and calls.

```python
show_wordcloud('ham')
```

On the other hand the most frequent words within normal messages are slang words and acronyms or words connected with some positive emotions.

In the next step we need to map labels to numerical values, lowercase messages, remove punctuation, tokenize the words and finally remove stop words.

```python
df['label'] = df['label'].map({'ham': 0, 'spam': 1})
df['message'] = df['message'].str.lower()
df['message'] = df['message'].apply(lambda x: ''.join([i for i in x if i not in string.punctuation]))
df['message'] = df['message'].str.split()
df['message'] = df['message'].apply(lambda row: [word for word in row if word not in STOPWORDS])
df.head()
```

      label
      message

      0
      0
      [go, jurong, point, crazy, available, bugis, n...

      1
      0
      [ok, lar, joking, wif, u, oni]

      2
      1
      [free, entry, 2, wkly, comp, win, fa, cup, fin...

      3
      0
      [u, dun, say, early, hor, u, c, already, say]

      4
      0
      [nah, dont, think, goes, usf, lives, around, t...

Now we need to split the dataset for training set (80%) and testing set (20%).

```python
train_set, test_set = train_test_split(df, test_size=0.2, random_state=0)
```

When we have our training set, we can create bags of words both for ham and spam messages.

```python
def create_bow(label):
    bow = {}
    for word_list in train_set[train_set['label'] == label]['message']:
        for word in word_list:
            if word in bow:
                bow[word] += 1
            else:
                bow[word] = 1
    return bow

ham_bow = create_bow(0)
spam_bow = create_bow(1)

ham_unique_words = len(ham_bow.keys())
ham_total_words = sum(ham_bow.values())
spam_unique_words = len(spam_bow.keys())
spam_total_words = sum(spam_bow.values())
```

The probability of message belonging to the appropriate class if it contains a given word can be calculated using Naive Bayes theorem:

\begin{equation}
P(C|W_{i}) = \frac{P(W_{i}|C)\cdot P(C)}{P(W_{i}|C_{1})\cdot P(C_{1}) + P(W_{i}|C_{2})\cdot P(C_{2})}
\end{equation}

In the first place we need to calculate the prior probability that a message belongs to the given class.

```python
def p_class(label):
    return len(train_set[train_set['label'] == label])/len(train_set)
```

The next step is defining the function that we can use to calculate the likelihood that given word appears in either spam or ham message. We will apply Laplace smoothing to avoid the situation when the given word doesn't appear in the bag of words resulting in likelihood being equal 0:

\begin{equation}
P(W_{i}|C) = \frac{count(W_{i}, C) + 1}{count(C) + |V| +1}
\end{equation}

```python
def p_word_class(word, label):
    if label == 0:
        likelihood = (ham_bow.get(word, 0) + 1)/(ham_total_words + ham_unique_words + 1)
    else:
        likelihood = (spam_bow.get(word, 0) + 1)/(spam_total_words + spam_unique_words + 1)
    return likelihood
```

Now we can create a function that calculates posterior probability, based on the first equation.

```python
def p_class_word(word, label):
    p_word_spam = p_word_class(word, 1)
    p_word_ham = p_word_class(word, 0)
    p_spam = p_class(1)
    p_ham = p_class(0)
    if label == 0:
        probability = (p_word_ham * p_ham)/(p_word_spam * p_spam + p_word_ham * p_ham)
    else:
        probability = (p_word_spam * p_spam)/(p_word_spam * p_spam + p_word_ham * p_ham)
    return probability
```

Finally we are ready to combine single probabilities so we can consider all the words in the message. To do so we will use the following equation:

\begin{equation}
P(C|W) = \frac{P(C|W_{1})\cdot P(C|W_{2})\cdots P(C|W_{n})}{P(C|W_{1})\cdot P(C|W_{2})\cdots P(C|W_{n}) + (1 - P(C|W_{1}))(1 - P(C|W_{2}))\cdots (1 - P(C|W_{n}))}
\end{equation}

```python
def p_class_words(words, label):
    probabilities = []
    for word in words:
        probabilities.append(p_class_word(word, label))
    probabilities_not = [1 - x for x in probabilities]
    total_probability = np.product(probabilities)/(np.product(probabilities) + np.product(probabilities_not))
    return total_probability
```

We will treat our implemented algorithm with the testing set to make predictions.

```python
y_pred = []
y_test = test_set['label'].values

for word_list in test_set['message']:
    probability = p_class_words(word_list, 1)
    y_pred.append(probability)

y_pred = np.asarray(y_pred)
```

The last step is the evaluation of the created classifier. To measure a performance of the model we will plot build a confusion matrix and plot ROC curve with AUC score.

```python
def build_cnf(y_test, y_pred):
    # Changing the continuous values of probabilities into discrete values of classes
    y_pred_disc = np.copy(y_pred)
    y_pred_disc[y_pred_disc > 0.5] = 1
    y_pred_disc[y_pred_disc <= 0.5] = 0

    cnf = metrics.confusion_matrix(y_test, y_pred_disc)
    accuracy = metrics.accuracy_score(y_test, y_pred_disc)
    precision = metrics.precision_score(y_test, y_pred_disc, average='macro')
    recall = metrics.recall_score(y_test, y_pred_disc, average='macro')

    print('Accuracy: ' + str(round(accuracy*100, 2)) + '%')
    print('Precision: ' + str(round(precision*100, 2)) + '%')
    print('Recall: ' + str(round(recall*100, 2)) + '%')

    heatmap = plt.imshow(cnf)
    ax = plt.gca()
    ax.set_xticks(np.arange(0, 2, 1))
    ax.set_yticks(np.arange(0, 2, 1))
    ax.set_xticklabels(['positive', 'negative'])
    ax.set_yticklabels(['positive', 'negative'])
    ax.set_xlabel('Predicted')
    ax.set_ylabel('Actual')
    ax.set_title('Confusion matrix')

    for i in range(np.shape(cnf)[0]):
        for j in range(np.shape(cnf)[1]):
            text = ax.text(j, i, cnf[i, j], ha='center', va='center')

    plt.setp(ax.get_yticklabels(), rotation=90, ha='center', rotation_mode='anchor')
    plt.colorbar(heatmap)
    plt.show()

def plot_roc(y_test, y_pred):
    fpr, tpr, _ = metrics.roc_curve(y_test, y_pred)
    auc = metrics.roc_auc_score(y_test, y_pred)
    plt.plot(fpr, tpr, label="ROC, AUC="+str(auc))
    plt.legend()
    plt.show()

build_cnf(y_test, y_pred)
plot_roc(y_test, y_pred)
```

As we can see the performance of implemented model is quite poor. Despite of accuracy reaching over 90%, recall and AUC score are fairly low which means that our model labeled a reasonable amount of spam messages as non-spam. Now let's create the classifier based on *scikit-learn* libraries and compare its performance with previously built model.

```python
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.naive_bayes import MultinomialNB

df_copy = df.copy()
df_copy['message'] = df['message'].apply(lambda word_list: ' '.join(word_list))

count_vect = CountVectorizer()
counts = count_vect.fit_transform(df_copy['message'])

X_train, X_test, Y_train, Y_test = train_test_split(counts, df_copy['label'], test_size=0.2, random_state=0)

model = MultinomialNB().fit(X_train, Y_train)
predictions = model.predict(X_test)

build_cnf(Y_test, predictions)
plot_roc(Y_test, predictions)
```

As expected *scikit-learn* model performs much better, reaching over 95% of accuracy and recall. Also AUC score is close to 1.


# Simulating Molecules using VQE

In this tutorial, we introduce the Variational Quantum Eigensolver (VQE), motivate its use, explain the necessary theory, and demonstrate its implementation in finding the ground state energy of molecules.

## Contents
1. [Introduction](#introduction)
2. [The Variational Method of Quantum Mechanics](#varmethod)
    1. [Mathematical Background](#backgroundmath)
    2. [Bounding the Ground State](#groundstate)
3. [The Variational Quantum Eigensolver](#vqe)
    1. [Variational Forms](#varforms)
    2. [Simple Variational Forms](#simplevarform)
    3. [Parameter Optimization](#optimization)
    4. [Example with a Single Qubit Variational Form](#example)
    5. [Structure of Common Variational Forms](#commonvarforms)
4. [VQE Implementation in Qiskit](#implementation)
    1. [Running VQE on a Statevector Simulator](#implementationstatevec)
    2. [Running VQE on a Noisy Simulator](#implementationnoisy)
5. [Problems](#problems)
6. [References](#references)

## Introduction
In many applications it is important to find the minimum eigenvalue of a matrix. For example, in chemistry, the minimum eigenvalue of a Hermitian matrix characterizing the molecule is the ground state energy of that system. In the future, the quantum phase estimation algorithm may be used to find the minimum eigenvalue. However, its implementation on useful problems requires circuit depths exceeding the limits of hardware available in the NISQ era. Thus, in 2014, Peruzzo *et al.* proposed VQE to estimate the ground state energy of a molecule using much shallower circuits [1].

Formally stated, given a Hermitian matrix $H$ with an unknown minimum eigenvalue $\lambda_{min}$, associated with the eigenstate $|\psi_{min}\rangle$, VQE provides an estimate $\lambda_{\theta}$ bounding $\lambda_{min}$:

\begin{align*}
    \lambda_{min} \le \lambda_{\theta} \equiv \langle \psi(\theta) |H|\psi(\theta) \rangle
\end{align*}

where $|\psi(\theta)\rangle$ is the eigenstate associated with $\lambda_{\theta}$. By applying a parameterized circuit, represented by $U(\theta)$, to some arbitrary starting state $|\psi\rangle$, the algorithm obtains an estimate $U(\theta)|\psi\rangle \equiv |\psi(\theta)\rangle$ on $|\psi_{min}\rangle$. The estimate is iteratively optimized by a classical controller changing the parameter $\theta$ minimizing the expectation value of $\langle \psi(\theta) |H|\psi(\theta) \rangle$.

## The Variational Method of Quantum Mechanics
### Mathematical Background

VQE is an application of the variational method of quantum mechanics. To better understand the variational method, some preliminary mathematical background is provided. An eigenvector, $|\psi_i\rangle$, of a matrix $A$ is invariant under transformation by $A$ up to a scalar multiplicative constant (the eigenvalue $\lambda_i$). That is,

\begin{align*}
    A |\psi_i\rangle = \lambda_i |\psi_i\rangle
\end{align*}

Furthermore, a matrix $H$ is Hermitian when it is equal to its own conjugate transpose.

\begin{align*}
    H = H^{\dagger}
\end{align*}

The spectral theorem states that the eigenvalues of a Hermitian matrix must be real. Thus, any eigenvalue of $H$ has the property that $\lambda_i = \lambda_i^*$. As any measurable quantity must be real, Hermitian matrices are suitable for describing the Hamiltonians of quantum systems. Moreover, $H$ may be expressed as

\begin{align*}
    H = \sum_{i = 1}^{N} \lambda_i |\psi_i\rangle \langle \psi_i |
\end{align*}

where each $\lambda_i$ is the eigenvalue corresponding to the eigenvector $|\psi_i\rangle$. Furthermore, the expectation value of the observable $H$ on an arbitrary quantum state $|\psi\rangle$ is given by

\begin{align}
    \langle H \rangle_{\psi} &\equiv \langle \psi | H | \psi \rangle
\end{align}

Substituting $H$ with its representation as a weighted sum of its eigenvectors,

\begin{align}
    \langle H \rangle_{\psi} = \langle \psi | H | \psi \rangle &= \langle \psi | \left(\sum_{i = 1}^{N} \lambda_i |\psi_i\rangle \langle \psi_i |\right) |\psi\rangle\\
    &= \sum_{i = 1}^{N} \lambda_i \langle \psi | \psi_i\rangle \langle \psi_i | \psi\rangle \\
    &= \sum_{i = 1}^{N} \lambda_i | \langle \psi_i | \psi\rangle |^2
\end{align}

The last equation demonstrates that the expectation value of an observable on any state can be expressed as a linear combination using the eigenvalues associated with $H$ as the weights. Moreover, each of the weights in the linear combination is greater than or equal to 0, as $| \langle \psi_i | \psi\rangle |^2 \ge 0$ and so it is clear that

\begin{align}
    \lambda_{min} \le \langle H \rangle_{\psi} = \langle \psi | H | \psi \rangle = \sum_{i = 1}^{N} \lambda_i | \langle \psi_i | \psi\rangle |^2
\end{align}

The above equation is known as the **variational method** (in some texts it is also known as the variational principle) [2]. It is important to note that this implies that the expectation value of any wave function will always be at least the minimum eigenvalue associated with $H$. Moreover, the expectation value of state $|\psi_{min}\rangle$ is given by $\langle \psi_{min}|H|\psi_{min}\rangle = \langle \psi_{min}|\lambda_{min}|\psi_{min}\rangle = \lambda_{min}$. Thus, as expected, $\langle H \rangle_{\psi_{min}}=\lambda_{min}$.

### Bounding the Ground State
When the Hamiltonian of a system is described by the Hermitian matrix $H$ the ground state energy of that system, $E_{gs}$, is the smallest eigenvalue associated with $H$. By arbitrarily selecting a wave function $|\psi \rangle$ (called an *ansatz*) as an initial guess approximating $|\psi_{min}\rangle$, calculating its expectation value, $\langle H \rangle_{\psi}$, and iteratively updating the wave function, arbitrarily tight bounds on the ground state energy of a Hamiltonian may be obtained.

## The Variational Quantum Eigensolver
### Variational Forms
A systematic approach to varying the ansatz is required to implement the variational method on a quantum computer. VQE does so through the use of a parameterized circuit with a fixed form. Such a circuit is often called a *variational form*, and its action may be represented by the linear transformation $U(\theta)$. A variational form is applied to a starting state $|\psi\rangle$ (such as the vacuum state $|0\rangle$, or the Hartree Fock state) and generates an output state $U(\theta)|\psi\rangle\equiv |\psi(\theta)\rangle$. Iterative optimization over $|\psi(\theta)\rangle$ aims to yield an expectation value $\langle \psi(\theta)|H|\psi(\theta)\rangle \approx E_{gs} \equiv \lambda_{min}$. Ideally, $|\psi(\theta)\rangle$ will be close to $|\psi_{min}\rangle$ (where 'closeness' is characterized by either state fidelity, or Manhattan distance) although in practice, useful bounds on $E_{gs}$ can be obtained even if this is not the case.

Moreover, a fixed variational form with a polynomial number of parameters can only generate transformations to a polynomially sized subspace of all the states in an exponentially sized Hilbert space. Consequently, various variational forms exist. Some, such as Ry and RyRz are heuristically designed, without consideration of the target domain. Others, such as UCCSD, utilize domain specific knowledge to generate close approximations based on the problem's structure. The structure of common variational forms is discussed in greater depth later in this document.

### Simple Variational Forms
When constructing a variational form we must balance two opposing goals. Ideally, our $n$ qubit variational form would be able to generate any possible state $|\psi\rangle$ where $|\psi\rangle \in \mathbb{C}^N$ and $N=2^n$. However, we would like the variational form to use as few parameters as possible. Here, we aim to give intuition for the construction of variational forms satisfying our first goal, while disregarding the second goal for the sake of simplicity.

Consider the case where $n=1$. The U3 gate takes three parameters, $\theta, \phi$ and $\lambda$, and represents the following transformation:

$$
\begin{align}
    U3(\theta, \phi, \lambda) = \begin{pmatrix}\cos(\frac{\theta}{2}) & -e^{i\lambda}\sin(\frac{\theta}{2}) \\ e^{i\phi}\sin(\frac{\theta}{2}) & e^{i\lambda + i\phi}\cos(\frac{\theta}{2}) \end{pmatrix}
\end{align}
$$

Up to a global phase, any possible single qubit transformation may be implemented by appropriately setting these parameters. Consequently, for the single qubit case, a variational form capable of generating any possible state is given by the circuit:

Moreover, this universal 'variational form' only has 3 parameters and thus can be efficiently optimized. It is worth emphasising that the ability to generate an arbitrary state ensures that during the optimization process, the variational form does not limit the set of attainable states over which the expectation value of $H$ can be taken. Ideally, this ensures that the minimum expectation value is limited only by the capabilities of the classical optimizer.

A less trivial universal variational form may be derived for the 2 qubit case, where two body interactions, and thus entanglement, must be considered to achieve universality. Based on the work presented by *Shende et al.* [3] the following is an example of a universal parameterized 2 qubit circuit:

Allow the transformation performed by the above circuit to be represented by $U(\theta)$. When optimized variationally, the expectation value of $H$ is minimized when $U(\theta)|\psi\rangle \equiv |\psi(\theta)\rangle \approx |\psi_{min}\rangle$. By formulation, $U(\theta)$ may produce a transformation to any possible state, and so this variational form may obtain an arbitrarily tight bound on two qubit ground state energies, only limited by the capabilities of the classical optimizer.

### Parameter Optimization
Once an efficiently parameterized variational form has been selected, in accordance with the variational method, its parameters must be optimized to minimize the expectation value of the target Hamiltonian. The parameter optimization process has various challenges. For example, quantum hardware has various types of noise and so objective function evaluation (energy calculation) may not necessarily reflect the true objective function. Additionally, some optimizers perform a number of objective function evaluations dependent on cardinality of the parameter set. An appropriate optimizer should be selected by considering the requirements of a application.

A popular optimization strategy is gradient decent where each parameter is updated in the direction yielding the largest local change in energy. Consequently, the number of evaluations performed depends on the number of optimization parameters present. This allows the algorithm to quickly find a local optimum in the search space. However, this optimization strategy often gets stuck at poor local optima, and is relatively expensive in terms of the number of circuit evaluations performed. While an intuitive optimization strategy, it is not recommended for use in VQE.

An appropriate optimizer for optimizing a noisy objective function is the *Simultaneous Perturbation Stochastic Approximation* optimizer (SPSA). SPSA approximates the gradient of the objective function with only two measurements. It does so by concurrently perturbing all of the parameters in a random fashion, in contrast to gradient decent where each parameter is perturbed independently. When utilizing VQE in either a noisy simulator or on real hardware, SPSA is a recommended as the classical optimizer.

When noise is not present in the cost function evaluation (such as when using VQE with a statevector simulator), a wide variety of classical optimizers may be useful. Two such optimizers supported by Qiskit Aqua are the *Sequential Least Squares Programming* optimizer (SLSQP) and the *Constrained Optimization by Linear Approximation* optimizer (COBYLA). It is worth noting that COBYLA only performs one objective function evaluation per optimization iteration (and thus the number of evaluations is independent of the parameter set's cardinality). Therefore, if the objective function is noise-free and minimizing the number of performed evaluations is desirable, it is recommended to try COBYLA.

### Example with a Single Qubit Variational Form

We will now use the simple single qubit variational form to solve a problem similar to ground state energy estimation. Specifically, we are given a random probability vector $\vec{x}$ and wish to determine a possible parameterization for our single qubit variational form such that it outputs a probability distribution that is close to $\vec{x}$ (where closeness is defined in terms of the Manhattan distance between the two probability vectors).

We first create the random probability vector in python:

```python
import numpy as np
np.random.seed(999999)
target_distr = np.random.rand(2)
# We now convert the random vector into a valid probability vector
target_distr /= sum(target_distr)
```

We subsequently create a function that takes the parameters of our single U3 variational form as arguments and returns the corresponding quantum circuit:

```python
from qiskit import QuantumCircuit, ClassicalRegister, QuantumRegister
def get_var_form(params):
    qr = QuantumRegister(1, name="q")
    cr = ClassicalRegister(1, name='c')
    qc = QuantumCircuit(qr, cr)
    qc.u3(params[0], params[1], params[2], qr[0])
    qc.measure(qr, cr[0])
    return qc
```

Now we specify the objective function which takes as input a list of the variational form's parameters, and returns the cost associated with those parameters:

```python
from qiskit import Aer, execute
backend = Aer.get_backend("qasm_simulator")
NUM_SHOTS = 10000

def get_probability_distribution(counts):
    output_distr = [v / NUM_SHOTS for v in counts.values()]
    if len(output_distr) == 1:
        output_distr.append(0)
    return output_distr

def objective_function(params):
    # Obtain a quantum circuit instance from the paramters
    qc = get_var_form(params)
    # Execute the quantum circuit to obtain the probability distribution associated with the current parameters
    result = execute(qc, backend, shots=NUM_SHOTS).result()
    # Obtain the counts for each measured state, and convert those counts into a probability vector
    output_distr = get_probability_distribution(result.get_counts(qc))
    # Calculate the cost as the distance between the output distribution and the target distribution
    cost = sum([np.abs(output_distr[i] - target_distr[i]) for i in range(2)])
    return cost
```

Finally, we create an instance of the COBYLA optimizer, and run the algorithm. Note that the output varies from run to run. Moreover, while close, the obtained distribution might not be exactly the same as the target distribution, however, increasing the number of shots taken will increase the accuracy of the output.

```python
from qiskit.aqua.components.optimizers import COBYLA

# Initialize the COBYLA optimizer
optimizer = COBYLA(maxiter=500, tol=0.0001)

# Create the initial parameters (noting that our single qubit variational form has 3 parameters)
params = np.random.rand(3)
ret = optimizer.optimize(num_vars=3, objective_function=objective_function, initial_point=params)

# Obtain the output distribution using the final parameters
qc = get_var_form(ret[0])
counts = execute(qc, backend, shots=NUM_SHOTS).result().get_counts(qc)
output_distr = get_probability_distribution(counts)

print("Target Distribution:", target_distr)
print("Obtained Distribution:", output_distr)
print("Output Error (Manhattan Distance):", ret[1])
print("Parameters Found:", ret[0])

```

    Target Distribution: [0.51357006 0.48642994]
    Obtained Distribution: [0.5053, 0.4947]
    Output Error (Manhattan Distance): 0.020140118738839197
    Parameters Found: [1.59321467 0.07436451 1.67267425]

### Structure of Common Variational Forms
As already discussed, it is not possible for a polynomially parameterized variational form to generate a transformation to any state. Variational forms can be grouped into two categories, depending on how they deal with this limitation. The first category of variational forms use domain or application specific knowledge to limit the set of possible output states. The second approach uses a heuristic circuit without prior domain or application specific knowledge.

The first category of variational forms exploit characteristics of the problem domain to restrict the set of transformations that may be required. For example, when calculating the ground state energy of a molecule, the number of particles in the system is known *a priori*. Therefore, if a starting state with the correct number of particles is used, by limiting the variational form to only producing particle preserving transformations, the number of parameters required to span the new transformation subspace can be greatly reduced. Indeed, by utilizing similar information from Coupled-Cluster theory, the variational form UCCSD can obtain very accurate results for molecular ground state energy estimation when starting from the Hartree Fock state. Another example illustrating the exploitation of domain-specific knowledge follows from considering the set of circuits realizable on real quantum hardware. Extant quantum computers, such as those based on super conducting qubits, have limited qubit connectivity. That is, it is not possible to implement 2-qubit gates on arbitrary qubit pairs (without inserting swap gates). Thus, variational forms have been constructed for specific quantum computer architectures where the circuits are specifically tuned to maximally exploit the natively available connectivity and gates of a given quantum device. Such a variational form was used in 2017 to successfully implement VQE for the estimation of the ground state energies of molecules as large as BeH$_2$ on an IBM quantum computer [4].

In the second approach, gates are layered such that good approximations on a wide range of states may be obtained. Qiskit Aqua supports three such variational forms: RyRz, Ry and SwapRz (we will only discuss the first two). All of these variational forms accept multiple user-specified configurations. Three essential configurations are the number of qubits in the system, the depth setting, and the entanglement setting. A single layer of a variational form specifies a certain pattern of single qubit rotations and CX gates. The depth setting says how many times the variational form should repeat this pattern. By increasing the depth setting, at the cost of increasing the number of parameters that must be optimized, the set of states the variational form can generate increases. Finally, the entanglement setting selects the configuration, and implicitly the number, of CX gates. For example, when the entanglement setting is linear, CX gates are applied to adjacent qubit pairs in order (and thus $n-1$ CX gates are added per layer). When the entanglement setting is full, a CX gate is applied to each qubit pair in each layer. The circuits for RyRz corresponding to `entanglement="full"` and `entanglement="linear"` can be seen by executing the following code snippet:

```python
from qiskit.circuit.library import EfficientSU2
entanglements = ["linear", "full"]
for entanglement in entanglements:
    form = EfficientSU2(num_qubits=4, entanglement=entanglement)
    if entanglement == "linear":
        print("=============Linear Entanglement:=============")
    else:
        print("=============Full Entanglement:=============")
    # We initialize all parameters to 0 for this demonstration
    display(form.draw(fold=100))
    print()
```

    =============Linear Entanglement:=============

    =============Full Entanglement:=============

Assume the depth setting is set to $d$. Then, RyRz has $n\times (d+1)\times 2$ parameters, Ry with linear entanglement has $2n\times(d + \frac{1}{2})$ parameters, and Ry with full entanglement has $d\times n\times \frac{(n + 1)}{2} + n$ parameters.

## VQE Implementation in Qiskit
This section illustrates an implementation of VQE using the programmatic approach. Qiskit Aqua also enables a declarative implementation, however, it reveals less information about the underlying algorithm. This code, specifically the preparation of qubit operators, is based on the code found in the Qiskit Tutorials repository (and as of July 2019, may be found at: https://github.com/Qiskit/qiskit-tutorials ).

The following libraries must first be imported.

```python
from qiskit.aqua.algorithms import VQE, NumPyEigensolver
import matplotlib.pyplot as plt
import numpy as np
from qiskit.chemistry.components.variational_forms import UCCSD
from qiskit.chemistry.components.initial_states import HartreeFock
from qiskit.circuit.library import EfficientSU2
from qiskit.aqua.components.optimizers import COBYLA, SPSA, SLSQP
from qiskit.aqua.operators import Z2Symmetries
from qiskit import IBMQ, BasicAer, Aer
from qiskit.chemistry.drivers import PySCFDriver, UnitsType
from qiskit.chemistry import FermionicOperator
from qiskit import IBMQ
from qiskit.aqua import QuantumInstance
from qiskit.ignis.mitigation.measurement import CompleteMeasFitter
from qiskit.providers.aer.noise import NoiseModel
```

### Running VQE on a Statevector Simulator
We demonstrate the calculation of the ground state energy for LiH at various interatomic distances. A driver for the molecule must be created at each such distance. Note that in this experiment, to reduce the number of qubits used, we freeze the core and remove two unoccupied orbitals. First, we define a function that takes an interatomic distance and returns the appropriate qubit operator, $H$, as well as some other information about the operator.

```python
def get_qubit_op(dist):
    driver = PySCFDriver(atom="Li .0 .0 .0; H .0 .0 " + str(dist), unit=UnitsType.ANGSTROM,
                         charge=0, spin=0, basis='sto3g')
    molecule = driver.run()
    freeze_list = [0]
    remove_list = [-3, -2]
    repulsion_energy = molecule.nuclear_repulsion_energy
    num_particles = molecule.num_alpha + molecule.num_beta
    num_spin_orbitals = molecule.num_orbitals * 2
    remove_list = [x % molecule.num_orbitals for x in remove_list]
    freeze_list = [x % molecule.num_orbitals for x in freeze_list]
    remove_list = [x - len(freeze_list) for x in remove_list]
    remove_list += [x + molecule.num_orbitals - len(freeze_list)  for x in remove_list]
    freeze_list += [x + molecule.num_orbitals for x in freeze_list]
    ferOp = FermionicOperator(h1=molecule.one_body_integrals, h2=molecule.two_body_integrals)
    ferOp, energy_shift = ferOp.fermion_mode_freezing(freeze_list)
    num_spin_orbitals -= len(freeze_list)
    num_particles -= len(freeze_list)
    ferOp = ferOp.fermion_mode_elimination(remove_list)
    num_spin_orbitals -= len(remove_list)
    qubitOp = ferOp.mapping(map_type='parity', threshold=0.00000001)
    qubitOp = Z2Symmetries.two_qubit_reduction(qubitOp, num_particles)
    shift = energy_shift + repulsion_energy
    return qubitOp, num_particles, num_spin_orbitals, shift
```

First, the exact ground state energy is calculated using the qubit operator and a classical exact eigensolver. Subsequently, the initial state $|\psi\rangle$ is created, which the VQE instance uses to produce the final ansatz $\min_{\theta}(|\psi(\theta)\rangle)$. The exact result and the VQE result at each interatomic distance is stored. Observe that the result given by `vqe.run(backend)['energy'] + shift` is equivalent the quantity $\min_{\theta}\left(\langle \psi(\theta)|H|\psi(\theta)\rangle\right)$, where the minimum is not necessarily the global minimum.

When initializing the VQE instance with `VQE(qubitOp, var_form, optimizer, 'matrix')` the expectation value of $H$ on $|\psi(\theta)\rangle$ is directly calculated through matrix multiplication. However, when using an actual quantum device, or a true simulator such as the `qasm_simulator` with `VQE(qubitOp, var_form, optimizer, 'paulis')` the calculation of the expectation value is more complicated. A Hamiltonian may be represented as a sum of a Pauli strings, with each Pauli term acting on a qubit as specified by the mapping being used. Each Pauli string has a corresponding circuit appended to the circuit corresponding to $|\psi(\theta)\rangle$. Subsequently, each of these circuits is executed, and all of the results are used to determine the expectation value of $H$ on $|\psi(\theta)\rangle$. In the following example, we initialize the VQE instance with `matrix` mode, and so the expectation value is directly calculated through matrix multiplication.

Note that the following code snippet may take a few minutes to run to completion.

```python
backend = BasicAer.get_backend("statevector_simulator")
distances = np.arange(0.5, 4.0, 0.1)
exact_energies = []
vqe_energies = []
optimizer = SLSQP(maxiter=5)
for dist in distances:
    qubitOp, num_particles, num_spin_orbitals, shift = get_qubit_op(dist)
    result = NumPyEigensolver(qubitOp).run()
    exact_energies.append(np.real(result.eigenvalues) + shift)
    initial_state = HartreeFock(
        num_spin_orbitals,
        num_particles,
        qubit_mapping='parity'
    )
    var_form = UCCSD(
        num_orbitals=num_spin_orbitals,
        num_particles=num_particles,
        initial_state=initial_state,
        qubit_mapping='parity'
    )
    vqe = VQE(qubitOp, var_form, optimizer)
    vqe_result = np.real(vqe.run(backend)['eigenvalue'] + shift)
    vqe_energies.append(vqe_result)
    print("Interatomic Distance:", np.round(dist, 2), "VQE Result:", vqe_result, "Exact Energy:", exact_energies[-1])

print("All energies have been calculated")
```

    Interatomic Distance: 0.5 VQE Result: -7.039673442796487 Exact Energy: [-7.03973252]
    Interatomic Distance: 0.6 VQE Result: -7.313308881704723 Exact Energy: [-7.31334583]
    Interatomic Distance: 0.7 VQE Result: -7.5008870360696545 Exact Energy: [-7.50092209]
    Interatomic Distance: 0.8 VQE Result: -7.630945411974594 Exact Energy: [-7.63097825]
    Interatomic Distance: 0.9 VQE Result: -7.7207810991039665 Exact Energy: [-7.72081241]
    Interatomic Distance: 1.0 VQE Result: -7.78221160063797 Exact Energy: [-7.7822424]
    Interatomic Distance: 1.1 VQE Result: -7.823568086767321 Exact Energy: [-7.82359928]
    Interatomic Distance: 1.2 VQE Result: -7.8506661274706175 Exact Energy: [-7.85069838]
    Interatomic Distance: 1.3 VQE Result: -7.867529589529125 Exact Energy: [-7.86756329]
    Interatomic Distance: 1.4 VQE Result: -7.876966252255704 Exact Energy: [-7.87700149]
    Interatomic Distance: 1.5 VQE Result: -7.88097950065464 Exact Energy: [-7.88101572]
    Interatomic Distance: 1.6 VQE Result: -7.881036613237316 Exact Energy: [-7.88107204]
    Interatomic Distance: 1.7 VQE Result: -7.878236970815989 Exact Energy: [-7.87826817]
    Interatomic Distance: 1.8 VQE Result: -7.873345207423448 Exact Energy: [-7.87344029]
    Interatomic Distance: 1.9 VQE Result: -7.8671374884820935 Exact Energy: [-7.86723396]
    Interatomic Distance: 2.0 VQE Result: -7.860068386104788 Exact Energy: [-7.86015321]
    Interatomic Distance: 2.1 VQE Result: -7.852535235082638 Exact Energy: [-7.85259583]
    Interatomic Distance: 2.2 VQE Result: -7.844839372545694 Exact Energy: [-7.84487909]
    Interatomic Distance: 2.3 VQE Result: -7.837212710920781 Exact Energy: [-7.83725797]
    Interatomic Distance: 2.4 VQE Result: -7.829853231704924 Exact Energy: [-7.829937]
    Interatomic Distance: 2.5 VQE Result: -7.822930322065164 Exact Energy: [-7.82307664]
    Interatomic Distance: 2.6 VQE Result: -7.816588425748401 Exact Energy: [-7.81679515]
    Interatomic Distance: 2.7 VQE Result: -7.810934384824721 Exact Energy: [-7.81116828]
    Interatomic Distance: 2.8 VQE Result: -7.80601482314384 Exact Energy: [-7.80622956]
    Interatomic Distance: 2.9 VQE Result: -7.801736715510548 Exact Energy: [-7.8019736]
    Interatomic Distance: 3.0 VQE Result: -7.798083502097215 Exact Energy: [-7.79836343]
    Interatomic Distance: 3.1 VQE Result: -7.795060180869786 Exact Energy: [-7.79534045]
    Interatomic Distance: 3.2 VQE Result: -7.792367370335087 Exact Energy: [-7.79283481]
    Interatomic Distance: 3.3 VQE Result: -7.789686196572474 Exact Energy: [-7.79077401]
    Interatomic Distance: 3.4 VQE Result: -7.7873755037333945 Exact Energy: [-7.7890889]
    Interatomic Distance: 3.5 VQE Result: -7.785627532109569 Exact Energy: [-7.78771697]
    Interatomic Distance: 3.6 VQE Result: -7.784375156000026 Exact Energy: [-7.78660376]
    Interatomic Distance: 3.7 VQE Result: -7.783495982816756 Exact Energy: [-7.78570291]
    Interatomic Distance: 3.8 VQE Result: -7.782885658211499 Exact Energy: [-7.78497559]
    Interatomic Distance: 3.9 VQE Result: -7.782466682970189 Exact Energy: [-7.78438961]
    All energies have been calculated

```python
plt.plot(distances, exact_energies, label="Exact Energy")
plt.plot(distances, vqe_energies, label="VQE Energy")
plt.xlabel('Atomic distance (Angstrom)')
plt.ylabel('Energy')
plt.legend()
plt.show()
```

Note that the VQE results are very close to the exact results, and so the exact energy curve is hidden by the VQE curve.

### Running VQE on a Noisy Simulator

Here, we calculate the ground state energy for H$_2$ using a noisy simulator and error mitigation.

First, we prepare the qubit operator representing the molecule's Hamiltonian:

```python
driver = PySCFDriver(atom='H .0 .0 -0.3625; H .0 .0 0.3625', unit=UnitsType.ANGSTROM, charge=0, spin=0, basis='sto3g')
molecule = driver.run()
num_particles = molecule.num_alpha + molecule.num_beta
qubitOp = FermionicOperator(h1=molecule.one_body_integrals, h2=molecule.two_body_integrals).mapping(map_type='parity')
qubitOp = Z2Symmetries.two_qubit_reduction(qubitOp, num_particles)
```

Now, we load a device coupling map and noise model from the IBMQ provider and create a quantum instance, enabling error mitigation:

```python
IBMQ.load_account()
provider = IBMQ.get_provider(hub='ibm-q')
backend = Aer.get_backend("qasm_simulator")
device = provider.get_backend("ibmq_vigo")
coupling_map = device.configuration().coupling_map
noise_model = NoiseModel.from_backend(device.properties())
quantum_instance = QuantumInstance(backend=backend,
                                   shots=1000,
                                   noise_model=noise_model,
                                   coupling_map=coupling_map,
                                   measurement_error_mitigation_cls=CompleteMeasFitter,
                                   cals_matrix_refresh_period=30)
```

Finally, we must configure the optimizer, the variational form, and the VQE instance. As the effects of noise increase as the number of two qubit gates circuit depth increase, we use a heuristic variational form (RYRZ) rather than UCCSD as RYRZ has a much shallower circuit than UCCSD and uses substantially fewer two qubit gates.

The following code may take a few minutes to run to completion.

```python
exact_solution = NumPyEigensolver(qubitOp).run()
print("Exact Result:", np.real(exact_solution.eigenvalues))
optimizer = SPSA(max_trials=100)
var_form = EfficientSU2(qubitOp.num_qubits, entanglement="linear")
vqe = VQE(qubitOp, var_form, optimizer=optimizer)
ret = vqe.run(quantum_instance)
vqe_result = np.real(ret['eigenvalue'])
print("VQE Result:", vqe_result)
```

    Exact Result: [-1.86712098]
    VQE Result: -1.80040360007339

When noise mitigation is enabled, even though the result does not fall within chemical accuracy (defined as being within 0.0016 Hartree of the exact result), it is fairly close to the exact solution.

## Problems
1. You are given a Hamiltonian $H$ with the promise that its ground state is close to a maximally entangled $n$ qubit state. Explain which variational form (or forms) is likely to efficiently and accurately learn the ground state energy of $H$. You may also answer by creating your own variational form, and explaining why it is appropriate for use with this Hamiltonian.
2. Calculate the number of circuit evaluations performed per optimization iteration, when using the COBYLA optimizer, the `qasm_simulator` with 1000 shots, and a Hamiltonian with 60 Pauli strings.
3. Use VQE to estimate the ground state energy of BeH$_2$ with an interatomic distance of $1.3$Å. You may re-use the function `get_qubit_op(dist)` by replacing `atom="Li .0 .0 .0; H .0 .0 " + str(dist)` with `atom="Be .0 .0 .0; H .0 .0 -" + str(dist) + "; H .0 .0 " + str(dist)` and invoking the function with `get_qubit_op(1.3)`. Note that removing the unoccupied orbitals does not preserve chemical precision for this molecule. However, to get the number of qubits  required down to 6 (and thereby allowing efficient simulation on most laptops), the loss of precision is acceptable. While beyond the scope of this exercise, the interested reader may use qubit tapering operations to reduce the number of required qubits to 7, without losing any chemical precision.

## References
1. Peruzzo, Alberto, et al. "A variational eigenvalue solver on a photonic quantum processor." *Nature communications* 5 (2014): 4213.
2. Griffiths, David J., and Darrell F. Schroeter. Introduction to quantum mechanics. *Cambridge University Press*, 2018.
3. Shende, Vivek V., Igor L. Markov, and Stephen S. Bullock. "Minimal universal two-qubit cnot-based circuits." arXiv preprint quant-ph/0308033 (2003).
4. Kandala, Abhinav, et al. "Hardware-efficient variational quantum eigensolver for small molecules and quantum magnets." Nature 549.7671 (2017): 242.

```python
import qiskit
qiskit.__qiskit_version__
```

    {'qiskit-terra': '0.14.2',
     'qiskit-aer': '0.5.2',
     'qiskit-ignis': '0.3.3',
     'qiskit-ibmq-provider': '0.7.2',
     'qiskit-aqua': '0.7.3',
     'qiskit': '0.19.6'}


To generate slides, execute the following command:

`jupyter nbconvert Projects.ipynb --to slides --no-prompt --TagRemovePreprocessor.remove_input_tags='["remove_cell"]' --SlidesExporter.reveal_theme=simple --post serve`

```python
import numpy as np
from scipy import linalg, signal
from matplotlib import pyplot as plt

rng = np.random.default_rng()
```

```python
def ceil(a):
    return int(a + 0.5)
```

```python
# Taken from https://codegolf.stackexchange.com/questions/69993/the-m%C3%B6bius-function
def mobius(n, d=1):
    return d % n and -mobius(d, n % d == 0) + mobius(n, d+1) or 1 // n
```

```python
def divisors(n):
    return [i for i in range(1, n // 2 + 1) if n % i == 0] + [n]
```

```python
MINIMUM_N = 2
MINIMUM_D = MINIMUM_N
```

# MTH 810 Projects
Luke Baumann
April 21st, 2021

*Trying out Jupyter slides. No spell check. Apologies in advance.

## Outline

0. Polynomial Class
1. Sieve of Eratosthenes for Irreducible Polynomials
2. Fast Number Theoretic Transform and Inverse
3. QR Code Overview

## Polynomial Class

Polynomials in $F_{p^\infty}$

Base Implemented Operations
1. `__add__`: Addition
2. `__neg__`: Additive inverse
3. `__lshift__`: Multiplication by x**n
4. `__mul__`: Multiplication by integers and polynomials
5. `__divmod__`: Long Division by integers and polynomials

Derived Operators
1. `__sub__`: Addition + Additive inverse -> Subtraction
2. `__floordiv__`: Quotient -> Long Division
3. `__mod__`: Remainder -> Long Division
4. `__pow__`: Multiplication -> Exponentiation

## Polynomial Class

Because $p$ is prime, for $a \in F_p$, $a^{-i} = a^{p - i - 1}$.

Likewise, if $P(x)$ is an irreducible polynomial, for $a \in F_{p^n} / P(x)$, $a^{-i} = a^{p^n - i - 1}$.

Thus, `__pow__` gives the multiplicative inverse.

Utility
1. `generate_polynomials`: Generates all polynomials up to a give degree
2. `generate_monomials`: Generate all monomials up to a give degree
3. `primitive_element`: Finding primitive element given a characteristic polynomial

```python
class PolynomialNP(object):
    def __init__(self, p, polynomial):
        self.p = p

        if isinstance(polynomial, int):
            degree = polynomial
            polynomial = np.zeros(degree + 1, dtype=np.int64)
        elif isinstance(polynomial, (tuple, list)):
            polynomial = np.array(polynomial)

        if np.any(polynomial > p):
            raise ValueError('polynomial has an entry not in alphabet p', polynomial)
        if not np.issubdtype(polynomial.dtype, np.integer):
            raise ValueError('polynomial coefficients must be of integer type', polynomial.dtype)

        polynomial = np.int64(polynomial.copy())

        self.polynomial = polynomial

    @classmethod
    def identity(cls, p):
        return cls(p, np.array([1], dtype=np.int64))

    @classmethod
    def zero(cls, p):
        return cls(p, np.array([], dtype=np.int64))

    def __hash__(self):
        """Needed so that polynomials can be put into a set among other things.
        Should not change polynomial after hashing unless you want bad things to happen
        """
        return hash(self.p) ^ hash(tuple(i for i in self.polynomial))

    def __eq__(self, other):
        """Needed so that polynomials can be put into a set among other things.
        Should not change polynomial after hashing unless you want bad things to happen
        """
        if not isinstance(other, PolynomialNP):
            return False
        return (self.degree == other.degree) and (self.p == other.p) and np.all(self.polynomial == other.polynomial)

    @property
    def degree(self):
        """The degree of the polynomial is one less than the number of terms"""
        return self.polynomial.size - 1

    def copy(self):
        """Returns a copy of the polynomial"""
        return PolynomialNP(self.p, self.polynomial.copy())

    def __add__(self, other):
        """Adds two polynomials together.
        The degree is the max degree of the two polynomials.
        Each term is mod added."""
        if isinstance(other, PolynomialNP):
            if self.p != other.p:
                raise ValueError('Can only add polynomials with the same alphabet', self.p, other.p)

            ret = np.zeros(max(self.degree + 1, other.degree + 1), dtype=np.int64)
            ret[:self.degree + 1] += self.polynomial
            ret[:other.degree + 1] += other.polynomial

            ret %= self.p

            return PolynomialNP(self.p, np.trim_zeros(ret, 'b'))

        else:
            return NotImplemented

        return ret

    def __lshift__(self, other):
        """Multiplies by x**other.
        New degree is old degree + 1 + other"""
        if not isinstance(other, int):
            return NotImplemented

        degree_increase = other
        ret = np.zeros(self.degree + 1 + degree_increase, dtype=np.int64)
        ret[degree_increase:] = self.polynomial

        return PolynomialNP(self.p, ret)

    def __mul__(self, other):
        """Multiplies polynomial by an integer or another polynomial.
        If multiplying by an integer, it must be within the alphabet and the
        term-by-term multiplication is mod p.
        If multiplying by a polynomial, multiply and shift the one polynomial
        by each factor and by each order in the other polynomial"""
        if isinstance(other, int):
            factor = other
            ret = self.polynomial * factor
            ret %= self.p

            ret = PolynomialNP(self.p, ret)

        elif isinstance(other, PolynomialNP):
            if self.p != other.p:
                raise ValueError('Can only multiply polynomials with the same alphabet', self.p, other.p)

            ret = np.zeros(self.degree + other.degree + 1, dtype=np.int64)

            outer_product = self.polynomial[:, np.newaxis] * other.polynomial[np.newaxis, :]

            for i in range(self.degree + 1):
                ret[i:i + other.degree + 1] += outer_product[i, :]

            ret = PolynomialNP(self.p, ret % self.p)

        else:
            return NotImplemented

        return ret

    __rmul__ = __mul__

    def __sub__(self, other):
        return self + -other

    def __neg__(self):
        return PolynomialNP(self.p, -self.polynomial % self.p)

    def __divmod__(self, other):
        """Calculates the quotient and remainder after division of by other"""

        # if dividing by a number, covert it to a polynomial of degree 0 and try again.
        if isinstance(other, int):
            return divmod(self, PolynomialNP(self.p, [other]))

        elif isinstance(other, PolynomialNP):
            if self.p != other.p:
                raise ValueError('Can only divmod polynomials with the same alphabet', self.p, other.p)

            # First, we want to multiply the top and bottom by a factor such that we are
            # dividing by a monomial so we find the factor here.
            for factor in range(1, self.p):
                if factor * other.polynomial[-1] % self.p == 1:
                    break
            if factor * other.polynomial[-1] % self.p != 1:
                raise ValueError('Something is wrong. Should have been able to find '
                                 'multiplicative inverse of {}'.format(other.polynomial[-1]))

            # Perform long division
            # We apply the factor at each iteration so that ret is the final remainder instead of
            # factor * remainder, requiring another division to get the remainder.
            ret = self.copy()
            quotient = np.zeros(ret.degree + 1, dtype=np.int64)
            while ret.degree >= other.degree:
                quotient[ret.degree - other.degree] = ret.polynomial[-1] * factor % self.p
                ret -= ret.polynomial[-1] * (factor * other) << ret.degree - other.degree

        else:
            return NotImplemented

        return PolynomialNP(self.p, quotient), ret

    def __floordiv__(self, other):
        """Division of a polynomial by other"""
        return divmod(self, other)[0]

    def __mod__(self, other):
        """Modulo of a polynomial by other"""
        return divmod(self, other)[1]

    def __pow__(self, other):
        """Raise a polynomial to an integer power"""
        if isinstance(other, int):
            if other < 0:
                raise ValueError('Must raise polynomial to positive power. '
                                'If you are trying to raise a field element '
                                'to a negative power, raise it to p**n - other - 1 '
                                'power instead.')

            ret = PolynomialNP.identity(self.p)
            for i in range(other):
                ret *= self

        else:
            return NotImplemented

        return ret

    def __repr__(self):
        """Represent the polynomial in a string"""
        ret = []
        for i in range(self.degree + 1):
            factor = self.polynomial[i]
            if factor != 0:
                ret += ['{factor}*x**{i}'.format(factor=factor, i=i)]

        if len(ret) == 0:
            return '0'
        else:
            return ' + '.join(reversed(ret))

    def __str__(self):
        """Print information about the polynomial and its representation"""
        ret = []
        ret += ['p: {}'.format(self.p)]
        ret += ['Degree: {}'.format(self.degree)]
        ret += [repr(self)]
        return '\n'.join(ret)

    @classmethod
    def generate_polynomials(cls, p, N):
        polynomial = np.zeros(N + 1, dtype=np.int64)
        for i in range(p**(N + 1)):
            for j in range(N + 1):
                i, polynomial[j] = divmod(i, p)

            yield cls(p, polynomial)

    @classmethod
    def generate_monomials(cls, p, N_min, N_max=None):
        if N_max is None:
            N_min, N_max = 0, N_min

        for n in range(N_min, N_max + 1):
            for i in range(p**n):
                polynomial = np.zeros(n, dtype=np.int64)
                for j in range(n):
                    i, polynomial[j] = divmod(i, p)

                yield (cls.identity(p) << n) + cls(p, polynomial)

    @classmethod
    def primitive_element(cls, p, n, characteristic_poly):
        d = p**n - 1

        for poly in cls.generate_polynomials(p, n):
            if cls.primitive_element_test(poly, p, n, characteristic_poly):
                return poly

        raise ValueError('There is always a primitive element if p is prime and the '
                         'characteristic polynomial is irreducible but none were found.')

    @classmethod
    def primitive_element_test(cls, poly, p, n, characteristic_poly):
        polys = set()

        for i in range(1, p**n):
            poly_power = poly**i % characteristic_poly

            if poly_power in polys:
                return False

            polys.add(poly_power)

        return len(polys) == p**n - 1
```

## Sieve of Eratosthenes for Irreducible Polynomials

Rudimentary method to find irreducible polynomials below a certain degree.

### Review: Sieve of Eratosthenes for Prime Numbers

Rudimentary method to find prime numbers below a certain number.

```python
def sieve_of_eratosthenes_primes(N):
    # All positive numbers except 1 which is defined to be not irreducible
    potential_primes = set(range(2, N + 1))

    for number in range(N):
        if number in potential_primes:
            # Remove all multiples of number
            potential_primes -= set(number * i
                                    for i in range(2, ceil(N / number) + 1))

    return potential_primes
```

```python
N = 100
print('Primes under {}'.format(N), sieve_of_eratosthenes_primes(100))
```

    Primes under 100 {2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67, 71, 73, 79, 83, 89, 97}

```python
# Used the sieve to find primes. Can be replaced by any function that returns a set of primes
primes = sieve_of_eratosthenes_primes
```

```python
# Finds the smallest prime divisor under prime_max
def smallest_prime_divisor(x, prime_max=None):
    if prime_max is None:
        prime_max = x

    remainder = None
    for x_1 in primes(prime_max):
        if x_1 > np.sqrt(x) and x_1 != x:
            continue

        x_2, remainder = divmod(x, x_1)
        if remainder == 0:
            break

    if remainder != 0:
        raise ValueError('No prime divisor under {}'.format(prime_max))

    return x_1, x_2
```

### Sieve of Eratosthenes for Irreducible Polynomials
#### Number of Irreducible Polynomials

Can confirm the number of irreducible polynomials by Gauss's formula where $p$ is the dictionary size, $n$ is the degree of the monomial, $D$ is the set of all positive divisors of $n$, and $\mu$ is Möbius number (1 if number of factors if even, otherwise, -1).

$$
I_{p^n} = \sum \limits_{d \in D} \mu \left( n / d \right) p^d
$$

```python
def I_pn(p, n):
    l = [mobius(d)*p**(n // d) for d in divisors(n)]
    return int(sum(l) // n)
```

```python
def sieve_of_eratosthenes_irreducible_polynomials(p, N):
    # All monomials except 1*x**0 which is defined to be not irreducible
    potential_irreducible_monomials  = set(PolynomialNP.generate_monomials(p, 1, N))

    for mono in PolynomialNP.generate_monomials(p, N):
        if mono in potential_irreducible_monomials:
            # Remove all multiples of poly
            potential_irreducible_monomials -= set(
                mono * mono_other
                for mono_other in PolynomialNP.generate_monomials(p, 1, N - mono.degree))

    # Place monomials into a dictionary for easy inspection
    ret = {degree: set() for degree in range(1, N + 1)}
    for mono in potential_irreducible_monomials:
        ret[mono.degree].add(mono)

    return ret
```

```python
p = 3
N = 5
irreducible_polynomials = sieve_of_eratosthenes_irreducible_polynomials(p, N)
for n, ip in irreducible_polynomials.items():
    print('Degree', n)
    print('    I_pn:', I_pn(p, n))
    print('    # Found:', len(ip))
    print()
print('Irreducible Polynomials of Degree 2')
print(irreducible_polynomials[2])
```

    Degree 1
        I_pn: 3
        # Found: 3

    Degree 2
        I_pn: 3
        # Found: 3

    Degree 3
        I_pn: 8
        # Found: 8

    Degree 4
        I_pn: 18
        # Found: 18

    Degree 5
        I_pn: 48
        # Found: 48

    Irreducible Polynomials of Degree 2
    {1*x**2 + 1*x**0, 1*x**2 + 1*x**1 + 2*x**0, 1*x**2 + 2*x**1 + 2*x**0}

```python
N = 50
p = 3
y_s = []
n_s = np.arange(1, N)
for n in n_s:
    y_s.append(I_pn(p, n) / p**n)

plt.figure(dpi=200)
plt.semilogy(n_s, 1 / n_s, lw=3, ls='--', label='Reference: $1/n$')
plt.semilogy(n_s, y_s, lw=2, ls='-', label='\n'.join(['Probability of generating an',
                                                      'irreducible polynomial randomly']))
plt.legend()

plt.xlabel('$n$')
plt.ylabel('Probability')
plt.title('\n'.join(['Probability of Irreducible Polynomials',
                    '$p = {}$, degree $n$'.format(p)]))
plt.show()
```

## Fast Number Theoretic Transform and Inverse

Fast computation of a transfrom applied to polynomials over a finite field.

### Why Transform

Certain operators and analysis are easier (simpler, faster) to do in a transformed domain than the original domain.

### Review: Discrete Fourier Transforms (DFT)

Invertibly transform a sequence $x_n \in \mathbb{C}$ into another sequence $X_k \in \mathbb{C}$ to perform certain operations easier.

### Incomplete List of Benefits

1. Finite differences become multiplication by $1 - e^{-j 2 \pi k / N}$.
2. Cummulative sums become division by $1 - e^{-j 2 \pi k / N}$.
3. Convolution becomes point-wise multiplication.
4. Harmonic analysis is more natural

### Review: DFT and IDFT

$$
\begin{align}
X_{k} =& \sum \limits _{n=0}^{N-1}x_{n} \left( e^{-j 2\pi /N} \right)^{kn} & X = W^+ x\\
x_{n} =& \frac{1}{K} \sum \limits _{k=0}^{K-1}X_{k} \left( e^{j 2\pi /K} \right)^{nk} & x = \frac{1}{K} W^- X
\end{align}
$$

Results in a Vandermonde matrix with $\gamma^+ = e^{-j 2\pi / N}$ and $\gamma^- e^{j 2\pi / K}$.

$$W^+=\begin{bmatrix}
1&1&1&1&\cdots &1\\
1&{\gamma^+} &{\gamma^+} ^{2}&{\gamma^+} ^{3}&\cdots &{\gamma^+} ^{N-1}\\1&{\gamma^+}^{2}&{\gamma^+}^{4}&{\gamma^+}^{6}&\cdots &{\gamma^+} ^{(N-1)2}\\1&{\gamma^+} ^{3}&{\gamma^+} ^{6}&{\gamma^+} ^{9}&\cdots &{\gamma^+}^{(N-1)3}\\\vdots &\vdots &\vdots &\vdots &\ddots &\vdots \\1&{\gamma^+} ^{K-1}&{\gamma^+} ^{2(K-1)}&{\gamma^+} ^{3(K-1)}&\cdots &{\gamma^+}^{(N-1)(K-1)}\end{bmatrix}$$

Similar for $W^-$.

### Review: Cooley-Tukey FFT Algorithm

Reexpress DFT of size $N = N_1 N_2$ into $N_1$ DFTs of size $N_2$ recursively, i.e., each DFT of size $N_2 = N_3 N_4$ into $N_3$ DFTs of size $N_4$, etc.

E.g., radix-2 special case where $N_1 = 2$ and $N_2 = N / 2$

$$
X_{k}=\underbrace {\sum \limits _{i=0}^{N_2-1}x_{2n} \left( e^{-j 2\pi / N_2} \right)^{kn}} _{\mathrm {DFT\;of\;even-indexed\;part\;of\;} x_{n}}{} + \left(e^{-j 2 \pi / N} \right)^k \underbrace {\sum \limits _{n=0}^{N_2-1}x_{2n+1} \left( e^{-j 2\pi / N_2} \right)^{kn}} _{\mathrm {DFT\;of\;odd-indexed\;part\;of\;} x_{n}}
$$

$$
\begin{align}
X_{k}=&E_{k}+\left( e^{-j 2\pi / N} \right)^k O_{k}\\
X_{k+N_2}=&E_{k}- \left(e^{-j 2\pi / N} \right)^k O_{k}
\end{align}
$$

Similar for IFFT.

```python
def vander_matmul(w, x):
    N = x.size
    W = np.vander(w, x.size, increasing=True)

    return W @ x

def dft_builtin(x_n):
    N = x_n.size
    return linalg.dft(N) @ x_n

def idft_builtin(X_k):
    N = X_k.size
    return np.conj(linalg.dft(N)) @ X_k / N

def dft(x_n):
    N = x_n.size
    w = np.exp(-1j * 2 * np.pi * np.arange(N) / N)
    return vander_matmul(w, x_n)

def idft(X_k, orig_K=None):
    N = X_k.size
    w = np.exp(1j * 2 * np.pi * np.arange(N) / N)

    return vander_matmul(w, X_k) / N
```

```python
x_n1 = rng.random(20)
X_k = dft(x_n1)
x_n2 = idft(X_k)

print(linalg.norm(x_n1 - x_n2) / linalg.norm(x_n1))
```

    2.1783269226295907e-15

```python
def fft_radix2(x_n):
    """
    Cooley-Tukey Algorithm for Radix-2.

    Performs an FFT recusively until the size is no longer a multiple of 2 and then reverts to a DFT.
    """
    N = x_n.size
    if N < MINIMUM_N:
        return dft(x_n)

    # Check to see if N is divisible by 2. If not, perform a DFT
    try:
        N_1, N_2 = smallest_prime_divisor(N, 2)
    except ValueError:
        return dft(x_n)

    gamma = np.exp(-1j * 2 * np.pi / N)
    twiddle = gamma**np.arange(N_2)

    even = fft_radix2(x_n[::N_1])
    odd = fft_radix2(x_n[1::N_1])

    twiddle_odd = twiddle * odd

    ret = np.zeros(N, dtype=np.complex128)
    ret[:N_2] = even + twiddle_odd
    ret[N_2:] = even - twiddle_odd

    return ret
```

```python
def ifft_radix2(X_k):
    """
    Cooley-Tukey Algorithm for Radix-2.

    Performs an IFFT recusively until the size is no longer a multiple of 2 and then reverts to a IDFT.
    """
    N = X_k.size
    if N < MINIMUM_N:
        return idft(X_k)

    # Check to see if K is divisible by 2. If not, perform a IDFT
    try:
        N_1, N_2 = smallest_prime_divisor(N, 2)
    except ValueError:
        return idft(X_k)

    gamma = np.exp(1j * 2 * np.pi / N)
    twiddle = gamma**np.arange(N_2)

    even = ifft_radix2(X_k[::N_1])
    odd = ifft_radix2(X_k[1::N_1])

    twiddle_odd = twiddle * odd

    ret = np.zeros(N, dtype=np.complex128)
    ret[:N_2] = even + twiddle_odd
    ret[N_2:] = even - twiddle_odd

    return ret / N_1
```

```python
N = 2**10
x_n = rng.random(N)
X_k = fft_radix2(x_n)
x_n1 = ifft_radix2(X_k)

print('FFT Error:', linalg.norm(X_k - np.fft.fft(x_n)) / linalg.norm(np.fft.fft(x_n)))
print('IFFT Error:', linalg.norm(x_n1 - np.fft.ifft(X_k)) / linalg.norm(np.fft.ifft(X_k)))
print('IFFT(FFT(x_n)) Error:', linalg.norm(x_n - x_n1) / linalg.norm(x_n))
```

    FFT Error: 4.201343978483622e-15
    IFFT Error: 4.174086104620431e-15
    IFFT(FFT(x_n)) Error: 7.020672682930469e-15

### Review: Cooley-Tukey FFT Algorithm

More general case where $N = N_1 N_2$ where $N$ is the number of samples (in both time and frequency) at each level of the recursion.

$$
\begin{align}
X_{k_1 N_2 + k_2}=& \sum \limits_{n_1=0}^{N_1 - 1} \left(e^{-j 2 \pi / N} \right)^{n_1 \left( N_2 k_1 + k_2 \right)} \left( \sum \limits _{n_2=0}^{N_2-1}x_{N_1 n_2 + n_1} \left( e^{-j 2\pi / N_2} \right)^{k_2 n_2} \right) \\
x_{n_1 N_2 + n_2}=& \frac{1}{N_1} \sum \limits_{k_1=0}^{N_1 - 1} \left(e^{j 2 \pi / N} \right)^{k_1 \left( N_2 n_1 + n_2 \right)} \left( \frac{N_1}{N} \sum \limits _{k_2=0}^{N_2-1}X_{N_1 k_2 + k_1} \left( e^{j 2\pi / N_2} \right)^{n_2 k_2} \right)
\end{align}
$$

```python
def fft(x_n):
    N = x_n.size
    if N < MINIMUM_N:
        return dft(x_n)

    N_1, N_2 = smallest_prime_divisor(N)

    gamma = np.exp(-1j * 2 * np.pi / N)

    parts = [fft(x_n[n_1::N_1]) for n_1 in range(N_1)]

    ret = np.zeros(N, dtype=np.complex128)
    for k_1 in range(N_1):
        ret[k_1 * N_2:(k_1 + 1) * N_2] = sum(
            gamma**(n_1 * (N_2 * k_1 + np.arange(N_2))) * part
            for n_1, part in enumerate(parts))

    return ret
```

```python
def ifft(X_k):
    N = X_k.size
    if N < MINIMUM_N:
        return idft(X_k)

    N_1, N_2 = smallest_prime_divisor(N)

    gamma = np.exp(1j * 2 * np.pi / N)

    parts = [ifft(X_k[k_1::N_1]) for k_1 in range(N_1)]

    ret = np.zeros(N, dtype=np.complex128)
    for n_1 in range(N_1):
        ret[n_1 * N_2:(n_1 + 1) * N_2] = sum(
            gamma**(k_1 * (N_2 * n_1 + np.arange(N_2))) * part
            for k_1, part in enumerate(parts))

    return ret / N_1
```

```python
N = 2 * 3 * 5 * 7 * 11
x_n = rng.random(N)
X_k = fft(x_n)
x_n1 = ifft(X_k)

print('FFT Error:', linalg.norm(X_k - np.fft.fft(x_n)) / linalg.norm(np.fft.fft(x_n)))
print('IFFT Error:', linalg.norm(x_n1 - np.fft.ifft(X_k)) / linalg.norm(np.fft.ifft(X_k)))
print('IFFT(FFT(x_n)) Error:', linalg.norm(x_n - x_n1) / linalg.norm(x_n))
```

    FFT Error: 3.632963287721464e-14
    IFFT Error: 2.8881688456579974e-14
    IFFT(FFT(x_n)) Error: 5.745776431090915e-14

```python
rng = np.random.default_rng()
N = 2**3 * 3 * 5 * 7
print('N', N)
x_n = signal.windows.gaussian(N, N / 10) * np.cos(2 * np.pi / N * N / 12 * np.arange(N))

X_k0 = dft(x_n)
x_n0 = idft(X_k0)

print('Radix-2 FFT')
X_k1 = fft_radix2(x_n)
x_n1 = ifft_radix2(X_k1)
print('    Frequency Domain Error:',
      linalg.norm(X_k1 - X_k0) / linalg.norm(X_k0))
print('    Time Domain Error:',
      linalg.norm(x_n1 - x_n) / linalg.norm(x_n))
print()

print('FFT')
X_k2 = fft(x_n)
x_n2 = ifft(X_k2)
print('    Frequency Domain Error:',
      linalg.norm(X_k1 - X_k0) / linalg.norm(X_k0))
print('    Time Domain Error:',
      linalg.norm(x_n1 - x_n) / linalg.norm(x_n))
print()

print('Numpy')
X_k3 = np.fft.fft(x_n)
x_n3 = np.fft.ifft(X_k3)
print('    Frequency Domain Error:',
      linalg.norm(X_k3 - X_k0) / linalg.norm(X_k0))
print('    Time Domain Error:',
      linalg.norm(x_n3 - x_n) / linalg.norm(x_n))
print()
```

    N 840
    Radix-2 FFT
        Frequency Domain Error: 1.2417472992382048e-13
        Time Domain Error: 2.830460714125949e-14

    FFT
        Frequency Domain Error: 1.2417472992382048e-13
        Time Domain Error: 2.830460714125949e-14

    Numpy
        Frequency Domain Error: 1.2606068718914065e-13
        Time Domain Error: 3.415706460783746e-16

```python
plt.figure(dpi=200)
plt.subplots_adjust(hspace=0.5)
plt.subplot(211)
plt.plot(x_n0.real, lw=2.0, label='IDFT')
plt.plot(x_n1.real, lw=1.5, label='Radix-2 IFFT')
plt.plot(x_n2.real, lw=1.0, label='IFFT')
plt.plot(x_n3.real, lw=0.5, label="Numpy's IFFT")
plt.xlabel('$n$')
plt.ylabel('$x_n$')
plt.legend(loc='upper right')

plt.subplot(212)
plt.semilogy(abs(X_k0), lw=2.0, label='DFT')
plt.semilogy(abs(X_k1), lw=1.5, label='Radix-2 FFT')
plt.semilogy(abs(X_k2), lw=1.0, label='FFT')
plt.semilogy(abs(X_k3), lw=0.5, label="Numpy's FFT")
plt.xlabel('$k$')
plt.ylabel(r'$\left| X_k \right|$')
plt.legend()

plt.show()
```

### Number Theoretic Transform (NTT)

Invertibly transform a sequence $a_j \in F_{p^n} / P(x)$ (or with slight modification, a ring of integers modulo $p$) into another sequence $A_i \in F_{p^n} / P(x)$ to perform certain operations easier.

### Incomplete List of Benefits

1. Convolution becomes point-wise multiplication.
    1. Multiplication of polynomials over $F_{p^n} / P(x)$
    2. Multiplication of integer polynomials
    3. Multiplication of very large integers
    4. Division of polynomials over $F_{p^n} / P(x)$
2. ***Decoding Reed-Solomon (and Reed-Muller) codes fast because their generator/parity check matrices are Vandermonde matrices of primitive elements.***

### NTT (and INTT) over Finite Fields

1. Field $F_{p^n} / P(x)$ where $p$ is prime and $n \in \mathbb{Z}^+$ and $P(x)$ is an order $n$ irreducible polynomial.
2. Multiplicative group ${F^*}_{p^n} / P(x)$ which is composed of nonzero elements of $F_{p^n} / P(x)$.
3. $d d^\prime = p^n - 1$.
4. $w$ is the primitive element of $F_{p^n} / P(x)$.
5. $a_j$ is a sequence of $d$ elements in $F_{p^n} / P(x)$.
5. $A_i$ is a sequence of $d$ elements in $F_{p^n} / P(x)$.

#### We use the Polynomial class to get elements of $F_{p^n} / P(x)$ by using modulo arithmetic on polynomials.

$$\begin{matrix}
A_i =& \sum \limits _{j = 0}^{d - 1} a_j \left( w^{d^\prime} \right)^{i j} & A = W^+ a & \gamma^+ = w^{d^\prime}\\
a_j =& -d^\prime \sum \limits _{i=0}^{d - 1} A_i \left( w^{-d^\prime} \right)^{j i} & a = -d^\prime W^- A & \gamma^- = w^{-d^\prime}
\end{matrix}$$

### DFT/NTT Comparison

| Discrete Fourier Transform                                                                          | Number Theoretic Transform                                                                          |
|-----------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------|
| $x \in \mathbb{C}$                                                                                | $a \in F_{p^n} / P(x)$                                                                              |
| $X_k = \sum_{n = 0}^{N - 1} x_n \left( e^{-j 2 \pi / N} \right)^{k n}$                              | $A_i = \sum \limits _{j = 0}^{d - 1} a_j \left( w^{d^\prime} \right)^{i j}$                          |
| $x_n = \frac{1}{N} \sum_{k = 0}^{N - 1} X_k \left( e^{j 2 \pi / N} \right)^{n k}$                   | $a_j = -d^\prime \sum \limits _{i = 0}^{d - 1} A_i \left( w^{-d^\prime} \right)^{j i}$              |
| $W^\pm$ is a Vandermonde matrix with $\gamma^+ = e^{-j 2 \pi / N}$ and $\gamma^- = e^{j 2 \pi / N}$ | $W^\pm$ is a Vandermonde matrix with $\gamma^+ = w^{d^\prime}$ and $\gamma^- = w^{-d^\prime}$ |
| $e^{-j 2 \pi / N}$ is the $N$<sup>th</sup> root-of-unity                                            | $w$ is the primitive element of ${F^*}_{p^n} / P(x)$                                                |
| Can be computed fast                                                                                | Can be computed fast                                                                                |

```python
def vander_polynomial(w, characteristic_poly):
    ret = {}
    d = len(w)
    for i in range(d):
        for j in range(d):
            ret[(i, j)] = w[i]**j

    return ret

def vander_polynomial_matmul(p, n, w, characteristic_poly, x_i):
    d = len(x_i)
    W = vander_polynomial(w, characteristic_poly)

    ret = [PolynomialNP(p, n) for _ in range(d)]
    for i in range(d):
        for j in range(d):
            ret[i] += W[(i, j)] * x_i[j]

    for i in range(d):
        ret[i] %= characteristic_poly

    return ret

def ntt(a_j, p, n, characteristic_poly):
    d = len(a_j)
    d_prime, remainder = divmod(p**n - 1, d)
    if remainder != 0:
        raise ValueError('d d_prime must equal p**n - 1')

    primitive_element = PolynomialNP.primitive_element(p, n, characteristic_poly)
    r = primitive_element**d_prime
    w = [r**i for i in range(d)]

    return vander_polynomial_matmul(p, n, w, characteristic_poly, a_j)

def intt(A_i, p, n, characteristic_poly):
    d = len(A_i)
    d_prime, remainder = divmod(p**n - 1, d)
    if remainder != 0:
        raise ValueError('d d_prime must equal p**n - 1')

    primitive_element = PolynomialNP.primitive_element(p, n, characteristic_poly)
    r = primitive_element**(p**n - d_prime - 1)
    w = [r**j for j in range(d)]

    return [-d_prime * a_j
            for a_j in vander_polynomial_matmul(p, n, w, characteristic_poly, A_i)]
```

### Cooley-Tukey FNTT Algorithm

Skipping the radix-2 specification, the general case is shown with $d = d_1 d_2$ where $d$ is the number of samples (in both domains) at each level of the recursion.

$$
\begin{align}
A_{i_1 d_2 + i_2}=& \sum \limits_{j_1=0}^{d_1 - 1} \left(w^{d^\prime} \right)^{j_1 \left( d_2 i_1 + i_2 \right)} \left( \sum \limits _{j_2=0}^{d_2-1} a_{d_1 j_2 + j_1} \left( w^{d^\prime} \right)^{i_2 j_2} \right) \\
a_{j_1 d_2 + j_2}=& d_1^{-1} \sum \limits_{i_1=0}^{d_1 - 1} \left(w^{-d^\prime} \right)^{i_1 \left( d_2 j_1 + j_2 \right)} \left( -d^\prime d_1 \sum \limits _{i_2=0}^{d_2-1} A_{d_1 i_2 + i_1} \left( w^{-d^\prime} \right)^{j_2 i_2} \right)
\end{align}
$$

## ***THIS IS THE MOST IMPORTANT RESULT FOR THIS WHOLE PROJECT***

```python
p = 3
n = 2

d_prime = 2
d, remainder = divmod(p**n - 1, d_prime)
if remainder != 0:
    raise ValueError('d d_prime must equal p**n - 1')

characteristic_poly = PolynomialNP(p, [2, 1, 1])
primitive_element = PolynomialNP.primitive_element(p, n, characteristic_poly)

a_j = [PolynomialNP(p, rng.integers(0, p, n)) for i in range(d)]
A_i = ntt(a_j, p, n, characteristic_poly)
a_j1 = intt(A_i, p, n, characteristic_poly)

print('INTT(NTT) Error:', [aj - aj1 for aj, aj1 in zip(a_j, a_j1)])
```

    INTT(NTT) Error: [0, 0, 0, 0]

```python
def fntt(a_j, p, n, characteristic_polynomial):
    d = len(a_j)
    # Base case, for small d, do naive NTT
    if d < MINIMUM_D:
        return ntt(a_j, p, n, characteristic_polynomial)

    d_prime, remainder = divmod(p**n - 1, d)
    if remainder != 0:
        raise ValueError('d d_prime must equal p**n - 1')

    d_1, d_2 = smallest_prime_divisor(d)

    w = PolynomialNP.primitive_element(p, n, characteristic_poly)
    r = w**d_prime

    parts = [fntt(a_j[j_1::d_1], p, n, characteristic_polynomial)
             for j_1 in range(d_1)]

    A_i = [PolynomialNP(p, n) for i in range(d)]
    for i_1 in range(d_1):
        for i_2 in range(d_2):
            A_i[i_1 * d_2 + i_2] += sum(
                (r**(j_1 * (d_2 * i_1 + i_2)) * part[i_2]
                for j_1, part in enumerate(parts)), PolynomialNP(p, n))

    for i in range(d):
        A_i[i] %= characteristic_poly

    return A_i
```

```python
def fintt(A_i, p, n, characteristic_polynomial):
    d = len(A_i)
    # Base case, for small d, do naive INTT
    if d < MINIMUM_D:
        return intt(A_i, p, n, characteristic_polynomial)

    d_prime, remainder = divmod(p**n - 1, d)
    if remainder != 0:
        raise ValueError('d d_prime must equal p**n - 1')

    d_1, d_2 = smallest_prime_divisor(d)

    w = PolynomialNP.primitive_element(p, n, characteristic_poly)
    r = w**(p**n - d_prime - 1)

    parts = [fintt(A_i[i_1::d_1], p, n, characteristic_polynomial)
             for i_1 in range(d_1)]

    a_j = [PolynomialNP(p, n) for j in range(d)]
    for j_1 in range(d_1):
        for j_2 in range(d_2):
            a_j[j_1 * d_2 + j_2] += sum(
                (r**(i_1 * (d_2 * j_1 + j_2)) * part[j_2]
                for i_1, part in enumerate(parts)), PolynomialNP(p, n))

    for j in range(d):
        a_j[j] = d_1**(p - 1 - 1) * a_j[j] % characteristic_poly

    return a_j
```

```python
p = 3
n = 2
d_prime = 2
d, remainder = divmod(p**n - 1, d_prime)
if remainder != 0:
    raise ValueError('d d_prime must equal p**n - 1')
characteristic_poly = PolynomialNP(p, [2, 1, 1])
primitive_element = PolynomialNP.primitive_element(p, n, characteristic_poly)

a_j = [PolynomialNP(p, rng.integers(0, p, n)) for i in range(d)]
A_i = fntt(a_j, p, n, characteristic_poly)
a_j1 = fintt(A_i, p, n, characteristic_poly)

print('FNTT Error:', [Ai - Ai1 for Ai, Ai1 in zip(A_i, ntt(a_j, p, n, characteristic_poly))])
print('FINTT Error:', [aj - aj1 for aj, aj1 in zip(a_j, intt(A_i, p, n, characteristic_poly))])
print('FINTT(FNTT) Error:', [aj - aj1 for aj, aj1 in zip(a_j, a_j1)])
```

    FNTT Error: [0, 0, 0, 0]
    FINTT Error: [0, 0, 0, 0]
    FINTT(FNTT) Error: [0, 0, 0, 0]

## My Open Questions

1. What is the analog to finite difference? $A_j \left(1 - \left( w^{d^\prime} \right)^j \right)$
2. What is the analog to cummulative sum? $A_j \left(1 - \left( w^{d^\prime} \right)^j \right)^{-1}$
3. What is the analog to harmonic analysis? The interpretation of $A_j$.

## QR Code Overview
Most of this is taken from Wikipedia but a very thorough tutorial is given here: https://www.thonky.com/qr-code-tutorial/

### QR Code key facts

Quick Response Code is 2D barcode (machine-readable, optical label) invented in 1994.

* Micro QR Code sizes: 11x11 to 17x17
* QR Code sizes: 21x21 to 177x177
* Can encode numeric data (0-9), alphanumeric (0-9, a-z, A-Z, " $%*+-./:", byte data, and Kanji characters)
* Four levels of Reed-Solomon error correction (L: 7%, M: 15%, Q: 25%, H: 30%)
* Max Micro QR Code data: 35 numeric, 21 alphanumeric, 15 bytes, 9 Kanji.
* Max QR Code data: 7089 numeric, 4296 alphanumeric, 2953 bytes, 1817 Kanji.
* Reflection, rotation, color inversion independent
* Introduce intentional errors to incorporate colors, logos, art, etc.

FYI: QR Codes flip the terminology for blocks and codewords.

The number of erasures and errors that can be corrected is given by $e + 2 t \le d - p$ where $e$ is the number of erasures, $t$ is the number of errors, $d$ is the number of error correcting codewords, and $p$ is the number of misdecode protection codewords.
Not covered in our course, $p$ is the number of extra codewords that can help detect errors but not correct them, $n - k = d + p$.

| Version | Total number<br/>of codewords | Error correction Level      | Number of error<br/>correction codewords | Value of $p$           | Number of error<br/>correction blocks                     | Error correction code</br>per block $(n, k, d)_{2^8}$                                                                                |
|---------|-------------------------------|-----------------------------|----------------------------------------------------|------------------------|-----------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------|
| M1      | 5                             | Error detection only        | 2                                                  | 2                      | 1                                                         | (5, 3, 0)                                                                                |
| M4      | 24                            | L<br/> M<br/> Q<br/>        | 8<br/> 10<br/> 14<br/>                             | 2<br/> 0<br/> 0        | 1                                                         | (24, 16, 3)<sup>a</sup><br/> (24, 14, 5)<br/> (24, 10, 7)                                                                                |
| 1       | 26                            | L<br/> M<br/> Q<br/> H<br/> | 7<br/> 10<br/> 13<br/> 17<br/>                     | 3<br/> 2<br/> 1<br/> 1 | 1                                                         | (26, 19, 2)<sup>a</sup><br/> (26, 16, 4)<sup>a</sup><br/> (26, 13, 6)<sup>a</sup><br/> (26, 9, 8)<sup>a</sup>                                  |
| 10      | 346                           | L<br/> M<br/> Q<br/> H<br/> | 72<br/> 130<br/> 192<br/> 224<br/>                 | 0                      | 2<br/> 2<br/> 4<br/> 1<br/> 6<br/> 2<br/> 6<br/> 2        | (86, 68, 9)<br/> (87, 69, 9)<br/> (69, 43, 13)<br/> (70, 44, 13)<br/> (43, 19, 12)<br/> (44, 20, 12)<br/> (43, 15, 14)<br/> (44, 16, 14)       |
| 20      | 1085                          | L<br/> M<br/> Q<br/> H<br/> | 224<br/> 416<br/> 600<br/> 700<br/>                | 0                      | 3<br/> 5<br/> 3<br/> 13<br/> 15<br/> 5<br/> 15<br/> 10    | (135, 107, 14)<br/> (136, 108, 14)<br/> (67, 41, 13)<br/> (68, 42, 13)<br/> (54, 24, 15)<br/> (55, 25, 15)<br/> (43, 15, 14)<br/> (44, 16, 14) |
| 40      | 3706                          | L<br/> M<br/> Q<br/> H<br/> | 750<br/> 1372<br/> 2040<br/> 2430<br/>             | 0                      | 19<br/> 6<br/> 18<br/> 31<br/> 34<br/> 34<br/> 20<br/> 61 | (148, 118, 15)<br/> (149, 119, 15)<br/> (75, 47, 14)<br/> (76, 48, 14)<br/> (54, 24, 15)<br/> (55, 25, 15)<br/> (45, 15, 15)<br/> (46, 16, 15) |

a: Error correction capacity is less than half the number of error correction codewords to reduce the probability of misdecodes.

## Thanks!

```python

```