# 量子振幅増幅

量子振幅増幅は、重ね合わせ状態におけるある特定の状態の振幅を増幅させるアルゴリズムです。
後に説明するグローバーのアルゴリズムの中核をなします。

## アルゴリズムの説明

### Amplitude amplification (振幅増幅)
振幅という言葉が出てきましたが、各状態の係数のことを**振幅**、各状態が測定される確率のことを**確率振幅**と言います。

以下の式を見てみます。

$$
H \lvert 0 \rangle \otimes H \lvert 0 \rangle = \frac{1}{2} (\lvert 00 \rangle + \lvert 01 \rangle + \lvert10 \rangle + \lvert 11 \rangle) \xrightarrow{\text{振幅増幅}} \lvert 00 \rangle
$$

この場合、矢印の左側では確率振幅は全て $1/4$ となり、
右側の状態は $00$ の確率振幅が $1$ となっています。

Amplitude amplification は言葉の通りこの確率振幅を上のように増幅させるアルゴリズムです。

以下の図を用いて具体的な計算を説明します。

参考: https://en.wikipedia.org/wiki/Grover%27s_algorithm

まずは記号の説明をします。
$\lvert s\rangle$ は任意の初期状態です。
ここでは具体例として、全状態の重ね合わせとして考えます。この初期状態は後に説明するグローバーのアルゴリズムで用いられます。

$$
\lvert s \rangle = \otimes^n  H  \lvert 0 \rangle =\frac{1}{\sqrt{2^n}}\sum^{2^n}_{x\in \{0, 1\}^n}\lvert x \rangle
$$

このうち、振幅増幅させたい状態を $\omega$ としています。

$$
\lvert \omega \rangle = \frac{1}{\sqrt{2^n}} \lvert 00...010...00\rangle
$$

$x$ 番目に1が入っているとします。

$\lvert s'\rangle$ は $\lvert s\rangle$ から $\omega$ を除いたベクトルです。

$$
\lvert s' \rangle = \lvert s \rangle - \lvert \omega \rangle = \lvert \omega^{\perp} \rangle
$$

$\lvert \omega \rangle$ に垂直なことがわかります。

$U_{\omega}$ は $\lvert s' \rangle$ を軸に $\psi$ を反転させる行列です。
すなわち $\psi$ を $-\phi$ 回転させます。

$$
U_{\omega} \lvert \psi \rangle = \cos(-\phi) \tilde{\lvert s' \rangle} + \sin(-\phi) \tilde{\lvert \omega \rangle} = \cos(\phi) \tilde{\lvert s' \rangle} - \sin(\phi) \tilde{\lvert \omega \rangle}
$$

$U_s$ は $\psi$ を $\lvert s \rangle$ を軸に反転させる行列です。

$$
U_s \lvert \psi \rangle = \cos \bigg\{ \frac{\theta}{2} + \big(\frac{\theta}{2} - \phi \big) \bigg\} \tilde{\lvert s' \rangle} + \sin \bigg\{ \frac{\theta}{2} + \big(\frac{\theta}{2} - \phi \big) \bigg\}\tilde{\lvert \omega \rangle} = \cos(\theta - \phi)\tilde{\lvert s' \rangle} + \sin(\theta - \phi) \tilde{\lvert \omega \rangle}
$$

### 概要
アルゴリズムの概要は以下のようになります。

1. $\lvert s \rangle$ を $U_{\omega}$ を用いて $\lvert s' \rangle$ で折り返す。

2. $U_{\omega}\lvert s \rangle$ を $U_s$ を用いて $\lvert s \rangle$ で折り返す。

以上の流れを詳しく説明していきます。

#### 1. $\lvert s' \rangle$ に関する折り返し
$\lvert s \rangle$ について上の図のように $\theta$ を用いて

$$
\lvert s \rangle = \cos\bigl(\frac{\theta}{2}\bigr) \tilde{\lvert s' \rangle} - \sin\bigl(\frac{\theta}{2}\bigr) \tilde{\lvert \omega \rangle}
$$

と定義します。

このとき

$$
\cos\bigl( \frac{\theta}{2} \bigr) = \sqrt{\frac{2^n-1}{2^n}},\ \ \ \  \sin\bigl( \frac{\theta}{2} \bigr) = \sqrt{\frac{1}{2^n}}
$$

と表せます。

$U_{\omega}$ で $\lvert s \rangle$ を $\lvert s' \rangle$ を軸に折り返します。
上の図から以下のようにかけます。

$$
U_{\omega} \lvert s \rangle = \cos\bigl(-\frac{\theta}{2}\bigr)\tilde{\lvert s' \rangle} + \sin\bigl(-\frac{\theta}{2}\bigr)\tilde{\lvert \omega \rangle} = \cos\bigl(\frac{\theta}{2}\bigr)\tilde{\lvert s' \rangle} - \sin\bigl(\frac{\theta}{2}\bigr)\tilde{\lvert \omega \rangle}
$$

この操作に関しては $\lvert \omega \rangle$ のみに作用しているので $U_{\omega}$ は上で述べた Oracle を表していることがわかります。

#### 2. $\lvert s \rangle$ に関する折り返し
$U_s$ で $U_{\omega}\lvert s \rangle$ を $\lvert s \rangle$ を軸に折り返します。

$$
U_s U_{\omega} \lvert s\rangle = U_s\biggl( \cos\bigl(-\frac{\theta}{2}\bigr)\tilde{\lvert s' \rangle} + \sin\bigl(-\frac{\theta}{2}\bigr)\tilde{\lvert \omega \rangle}  \biggr)
$$

ここで $2\theta$ 回転させれば良いので

$$
U_s U_{\omega} \lvert s\rangle =  \cos\bigl(\frac{3}{2}\theta\bigr)\tilde{\lvert s' \rangle} + \sin\bigl(\frac{3}{2}\theta\bigr)\tilde{\lvert \omega \rangle}
$$

具体的に $\cos$ と $\sin$ を求めると加法定理から

$$
\cos \frac{3}{2}\theta = \bigl( 1-\frac{4}{2^n} \bigr) \sqrt{\frac{2^n-1}{2^n}},\ \ \ \ \sin \frac{3}{2}\theta = \bigl( 3-\frac{4}{2^n} \bigr) \sqrt{\frac{1}{2^n}}
$$

よって、$\lvert s' \rangle$, $\lvert \omega \rangle$ を用いると

$$
U_s U_{\omega} \lvert s\rangle =  \bigl(1 - \frac{4}{2^n}\bigr)\lvert s' \rangle + \bigl(3 - \frac{4}{2^n}\bigr)\lvert \omega \rangle
$$

この操作によって $2^n$ 個の振幅のうち $\lvert \omega \rangle$ が他のよりも約３倍大きくなりました。
以上で振幅増幅させることができました。

## 回路の実装

これをblueqatで実装してみましょう。

2量子ビットの量子振幅増幅を考えます。
2量子ビットの状態は、00,01,10,11の4通りです。その中から特定の状態を増幅したいと思います。
増幅したい状態は上記の $\lvert \omega \rangle$ に対応します。

#### 1. $\lvert s \rangle$ を $U_{\omega}$ を用いて $\lvert s' \rangle$ で折り返す。
まず、 $\lvert \omega \rangle$ と垂直な $\lvert s' \rangle$ を軸に、状態を反転させるゲート $U_{\omega}$ を実現する必要があります。
そのためにはゲート操作をつかって、「解に対応する状態ベクトルだけに-1がかかる対角行列」を数学的に作ります。
ここではHゲート、Sゲート、CZゲートを用いれば可能です。

各回路を一つ一つ見ていきましょう。.run_with_sympy_unitary()を実行することで、回路のユニタリ行列を確認することができます。
CZゲートの対角行列から始めて、それを変更しながら回路を作っていきます。

```python
from blueqat import Circuit
```

```python
'''
#marking on 11

-------*-----
-------Z-----
'''

Circuit(2).cz[0,1].run_with_sympy_unitary()
```

$\displaystyle \left[\begin{matrix}1 & 0 & 0 & 0\\0 & 1 & 0 & 0\\0 & 0 & 1 & 0\\0 & 0 & 0 & -1\end{matrix}\right]$

```python
'''
#marking on 01

----S--*--S---
-------Z-------
'''

Circuit(2).s[0].cz[0,1].s[0].run_with_sympy_unitary()
```

$\displaystyle \left[\begin{matrix}1 & 0 & 0 & 0\\0 & -1 & 0 & 0\\0 & 0 & 1 & 0\\0 & 0 & 0 & 1\end{matrix}\right]$

```python
'''
#marking on 10

--------*------
----S--Z--S---
'''

Circuit(2).s[1].cz[0,1].s[1].run_with_sympy_unitary()
```

$\displaystyle \left[\begin{matrix}1 & 0 & 0 & 0\\0 & 1 & 0 & 0\\0 & 0 & -1 & 0\\0 & 0 & 0 & 1\end{matrix}\right]$

```python

'''
#00

----S--*--S--
----S--Z--S--
'''

Circuit(2).s[:].cz[0,1].s[:].run_with_sympy_unitary()
```

$\displaystyle \left[\begin{matrix}1 & 0 & 0 & 0\\0 & -1 & 0 & 0\\0 & 0 & -1 & 0\\0 & 0 & 0 & -1\end{matrix}\right]$

最後の結果は符号が反転しています。このような場合には、すべてのマイナス記号をプラスに、プラス記号をマイナスに反転させるグローバル位相を考えることができます。

#### 2. $U_{\omega}\lvert s \rangle$ を $U_s$ を用いて $\lvert s \rangle$ で折り返す。

$U_s$ については以下のように考えます。

$\lvert s \rangle$ の定義から $\lvert s \rangle = \lvert s' \rangle + \lvert \omega \rangle$ と分けて考えると、$U_{\omega}$ はシンプルに

$$
U_{\omega} (\lvert s' \rangle + \lvert \omega \rangle) = \lvert s' \rangle - \lvert \omega \rangle
$$

となります。
つまりZ、CZゲートなどのように特定の状態だけ符号を変えるゲートを用いれば良いことがわかります。

$U_s$ は上の図から幾何的に考えると以下のようにかけます。

$$
\begin{align}
U_s U_{\omega} \lvert s \rangle &= 2(\langle s \lvert U_{\omega} \rvert s \rangle \rvert s \rangle - U_{\omega}\lvert s \rangle) + U_{\omega}\lvert s \rangle \\
&= 2\lvert s \rangle \langle s \lvert U_{\omega} \rvert s \rangle - U_{\omega}\lvert s \rangle \\
&= (2\lvert s\rangle \langle s\rvert - I) U_{\omega}\lvert s \rangle
\end{align}
$$

よって $U_s = 2\lvert s\rangle \langle s\rvert - I$ となります。

さらに初期状態 $\lvert s \rangle = \otimes^n  H  \lvert 0 \rangle$ とした場合、 $U_s$ は以下のように分解できます。

$$
2\lvert s\rangle \langle s\rvert - I = 2H^{\otimes n}\lvert 0^n\rangle \langle 0^n\rvert H^{\otimes n} - I = H^{\otimes n} (2\lvert 0^n\rangle \langle 0^n\rvert - I) H^{\otimes n}\ \ \ \ (\lvert 0^n \rangle = \lvert 00\cdots 00 \rangle)
$$

ここで $2\lvert 0^n\rangle \langle 0^n\rvert - I$ に関して

$$
2\lvert 0^n\rangle \langle 0^n\rvert - I =
    \begin{pmatrix}
      -1 & 0 & 0 & \ldots & 0 & 0 \\
      0 & 1 & 0 & \ldots & 0 & 0 \\
      0 & 0 & 1 & \ldots & 0 & 0 \\
      \vdots & \vdots & \vdots & \ddots & \vdots & \vdots \\
      0 & 0 & 0 & \ldots & 1 & 0 \\
      0 & 0 & 0 & \ldots & 0 & 1
    \end{pmatrix}
$$

と表せます。

これは

$$
XZX =
    \begin{pmatrix}
      -1 & 0 \\
      0 & 1
    \end{pmatrix}
$$

の性質から

$$
2\lvert 0^n\rangle \langle 0^n\rvert - I = X^{\otimes n}C^n ZX^{\otimes n}
$$

とかけます。

こちらをBlueqatに直してみます。

```python
'''
--H-X-*-X-H--
--H-X-Z-X-H--
'''

Circuit(2).h[:].x[:].cz[0,1].x[:].h[:].run_with_sympy_unitary()
```

$\displaystyle \left[\begin{matrix}\frac{1}{2} & - \frac{1}{2} & - \frac{1}{2} & - \frac{1}{2}\\- \frac{1}{2} & \frac{1}{2} & - \frac{1}{2} & - \frac{1}{2}\\- \frac{1}{2} & - \frac{1}{2} & \frac{1}{2} & - \frac{1}{2}\\- \frac{1}{2} & - \frac{1}{2} & - \frac{1}{2} & \frac{1}{2}\end{matrix}\right]$

以上から振幅増幅用オラクルをゲートで書き直すことができました。

では、実際の回路の実装です。

```python
#振幅増幅反転
a = Circuit(2).h[:].x[:].cz[0,1].x[:].h[:].m[:]

'''
#00回路
--H--S--*--S----H-X-*-X-H--
--H--S--Z--S----H-X-Z-X-H--
'''

(Circuit(2).h[:].s[:].cz[0,1].s[:] + a).run(shots=100)
```

    Counter({'00': 100})

```python
'''
#01回路
--H-----*-------H-X-*-X-H--
--H--S--Z--S---H-X-Z-X-H--
'''

(Circuit(2).h[:].s[1].cz[0,1].s[1] + a).run(shots=100)
```

    Counter({'01': 100})

```python
'''
#10回路
--H--S--*--S----H-X-*-X-H--
--H-----Z--------H-X-Z-X-H--
'''
(Circuit(2).h[:].s[0].cz[0,1].s[0] + a).run(shots=100)
```

    Counter({'10': 100})

```python
'''
#11回路
--H-----*-------H-X-*-X-H--
--H-----Z-------H-X-Z-X-H--
'''
(Circuit(2).h[:].cz[0,1] + a).run(shots=100)
```

    Counter({'11': 100})

以上より、特定の状態が測定される確率を増幅できました。

```python

```


# Bernstein-Vazirani algorithm(ベルンシュタイン・ヴァジラニ アルゴリズム)（概要）

Bernstein-Vazirani algorithm を説明します。

このアルゴリズムでは $x$ を入力とし、内部に持つビット列 $a$ を用いて出力を計算する次のような関数を考えます。

$$
f_a(x) = (a\cdot x) \bmod2 = (\sum_i a_i x_i) \bmod2
$$

目的は、$f_a(x)$ の出力からビット列 $a$ を求めることです。

ただし、$a, x$ はそれぞれ $n$ビット列 $a = a_0 a_1... a_{n-1}$, $x = x_0 x_1... x_{n-1}$ と表します。

$a$ が4ビット列 $1001$ の場合を考えましょう。
古典計算における愚直な求め方は、$x=1000, 0100, 0010, 0001$ のように1つのビットだけを1とし、他を0とした $x$ を順に入力します。
$0 \cdot 0 = 0 \cdot 1  = 0$ より $x_i=0$としたビットは結果に影響しないため、$x_i = 1$ とした1ビットずつ $a_i$ の値を決定することができます。

$a=1001$ なる $f_a(x)$ を、4通りの $x$ について計算すると以下のようになります。

$(1001 \cdot 1000) \bmod2 = 1$
$(1001 \cdot 0100) \bmod2 = 0$
$(1001 \cdot 0010) \bmod2 = 0$
$(1001 \cdot 0001) \bmod2 = 1$

以上の出力より $a=1001$ を得ることができます。

Bernstein-Vazirani algorithm は量子の重ね合わせを用いて1度の計算で $a$ を求めます。

$a=1001$ の場合の具体的な回路は次の通りです。

$a_i=1$ である量子ビット $i$ をそれぞれ制御量子ビットとした、$CX$ゲートを作用させます。
ターゲット量子ビットは全て4番目の補助量子ビットです。

それぞれの状態を確認します。

$$
\begin{align}
\lvert \psi_1\rangle &= \biggl(\otimes^4 H\lvert 0\rangle \biggr)\otimes H \lvert 1\rangle \\
&= \frac{1}{\sqrt{2}}(\lvert 0\rangle + \lvert 1\rangle) \otimes \frac{1}{\sqrt{2}}(\lvert 0\rangle + \lvert 1\rangle) \otimes \frac{1}{\sqrt{2}}(\lvert 0\rangle + \lvert 1\rangle) \otimes \frac{1}{\sqrt{2}}(\lvert 0\rangle + \lvert 1\rangle) \otimes \frac{1}{\sqrt{2}}(\lvert 0\rangle - \lvert 1\rangle)
\end{align}
$$

次に $\lvert \psi_2\rangle$ の状態を確認します。
$CX$ゲートを作用させる0番目と4番目の量子ビットだけに注目してみましょう。

$$
\begin{align}
\lvert \psi_1\rangle_{04} &= \frac{1}{\sqrt{2}}(\lvert 0\rangle_0 + \lvert 1\rangle_0) \otimes \frac{1}{\sqrt{2}}(\lvert 0\rangle_4 - \lvert 1\rangle_4) \\
&= \lvert +\rangle_0 \otimes \lvert -\rangle_4
\end{align}
$$

$$
\begin{align}
\lvert \psi_2\rangle_{04} &= \frac{1}{\sqrt{2}}\lvert 0\rangle_0 \otimes \frac{1}{\sqrt{2}}(\lvert 0\rangle_4 - \lvert 1\rangle_4) + \frac{1}{\sqrt{2}}\lvert 1\rangle_0 \otimes \frac{1}{\sqrt{2}}(\lvert 1\rangle_4 - \lvert 0\rangle_4) \\
&= \frac{1}{\sqrt{2}}\lvert 0\rangle_0 \otimes \frac{1}{\sqrt{2}}(\lvert 0\rangle_4 - \lvert 1\rangle_4) - \frac{1}{\sqrt{2}}\lvert 1\rangle_0 \otimes \frac{1}{\sqrt{2}}(\lvert 0\rangle_4 - \lvert 1\rangle_4) \\
&= \frac{1}{\sqrt{2}}(\lvert 0\rangle_0 - \lvert 1\rangle_0 ) \otimes \frac{1}{\sqrt{2}}(\lvert 0\rangle_4 - \lvert 1\rangle_4) \\
&=  \lvert -\rangle_0 \otimes \lvert -\rangle_4
\end{align}
$$

0番目と4番目の量子ビットに$CX$ゲートを作用させることで、0番目の量子ビットの位相を反転させて$\lvert + \rangle \to \lvert - \rangle$と変化させました。
同様に3番目と4番目の量子ビットに$CX$ゲートを作用させた状態 $\lvert \psi_3\rangle$ は次の通りです。

$$
\lvert \psi_3\rangle = \frac{1}{\sqrt{2}}(\lvert 0\rangle - \lvert 1\rangle) \otimes \frac{1}{\sqrt{2}}(\lvert 0\rangle + \lvert 1\rangle) \otimes \frac{1}{\sqrt{2}}(\lvert 0\rangle + \lvert 1\rangle) \otimes \frac{1}{\sqrt{2}}(\lvert 0\rangle - \lvert 1\rangle) \otimes \frac{1}{\sqrt{2}}(\lvert 0\rangle - \lvert 1\rangle)
$$

最後にHゲートを作用させます。

$$
\lvert \psi_4\rangle = \lvert 1\rangle \otimes \lvert 0\rangle \otimes \lvert 0\rangle \otimes \lvert 1\rangle \otimes \frac{1}{\sqrt{2}}(\lvert 0\rangle - \lvert 1\rangle)
$$

0番目から3番目の量子ビットを測定すると、$1001$が測定結果として得られます。
以上で$a$を求めることができました。

これをblueqatで実装してみましょう。

```python
from blueqat import Circuit
import numpy as np
```

まず、オラクル$U_f$を作用させる関数を用意します。

```python
def Uf(c, a):
    N = len(a)
    for i, val in enumerate(list(a)):
        if val == '1':
            c.cx[i, len(a)]
```

以下がアルゴリズム本体です。
最初に、求めたい $a$ を乱数で決めます。

オラクルを用いた量子回路の出力結果から $a$ を求め、正解かを確認します。

```python
n = 4

a = ''
for i in range(n):
    a += str(np.random.randint(2))

c = Circuit(n + 1)
c.x[n].h[:]
Uf(c, a)
c.h[:].m[:]
res = c.run(shots = 1000)

print(res)

if [arr[:n] for arr in res.keys()] == [a]:
    print("OK")
else:
    print("incorrect")
```

    Counter({'11001': 1000})
    OK

以上より、Bernstein-Vazirani algorithm によってオラクルが内部的に持つ $a$ を求めることができました。

```python

```


# "Spin Glass Models 5: Applications and Extensions"
> "In this blog post we will take another look at spin-glasses, in particular we look at some of the areas where spin-glass conecepts have been used. In particular we will look at applications and extensions to our existing methods."
- toc: true
- author: Lewis Cole (2020)
- branch: master
- badges: false
- comments: false
- categories: [Spin-Glass, Magnet, Markov-Random-Field, Hopfield-Network, Potts-Model, NK-Model, NKCS-Model]
- hide: false
- search_exclude: false
- image: https://github.com/lewiscoleblog/blog/raw/master/images/spin-glass/hopfield.png

```python
#hide
import warnings
warnings.filterwarnings('ignore')
```

___
This is the fifth blog post in a series - you can find the previous blog post [here](https://lewiscoleblog.com/spin-glass-models-4)
___

If you have read my previous few blog posts you will have some appreciation for spin-glasses and some models used to study and simulate them. In this blog post we will look a few applications and extensions of spin-glasses.

## Potts Spin-Glass Model
The Potts spin-glass model is a generalization of the Ising model we looked at previously. Whereas the Ising model allows only "up" or "down" spins the Potts model is a "q-spin" model where spins can take any of $q$ possible values. The Hamiltonian is typically specified as:
$$H = - \sum_{x,y} J_{xy} \delta(\sigma_x \sigma_y) - \sum_x h_x \sigma_x $$
Where $\delta(.)$ represents a Kronecker delta function. For $q=2$ this is equivalent to the Ising model.

Another representation of a Potts model is the vector Potts model, here spins take values as defined as angles:
$$ \theta_i = \frac{2 \pi i}{q} $$
For $i$ between 1 and $q$. The Hamiltonian can then be defined as:
$$H = - \sum_{x,y} J_{xy} cos(\sigma_x  - \sigma_y) - \sum_x h_x \sigma_x $$
The case for $q \to \infty$ is called the XY-model.

On a 2d square lattice we find that a phase transition exists for $q>4$ for $q\leq 4$ there is a 2nd order phase transition (as with the Ising model $q=2$). The techniques and simulation procedures discussed previously can be adapted to the Potts model with minimal change.

Other generalizations of this model have been used in various physical and biological applications. The Potts model has also been used in image processing.

## Markov-Random-Field
The Ising model can also be thought of as the simplest example of a Markov-Random-Field (MRF). The theory and application of these could easily take a blog post (or more) by itself. Instead we will just look at some basic definitions and the major theorems.

A Markov-Random-Field is an example of a probabilistic graphical model (PGM). We define a MRF via a set of random variables and an undirected graph: $(X, G)$. Unlike other some other types of PGM an MRF graph is undirected and there is no requirement for the graph to be acyclic. For a given graph $G = (V,E)$ with specified vertices and edges, a set of random variables $(X_v)_{v \in V}$ forms a Markov-Random-Field over G if they satisfy the folowing properties:
1. **Pairwise Markov Property:** any two non-adjacent variables are conditionally independent given all other variables:
$$X_u \perp X_v | X_{V / \{u,v\}}$$
2. **Local Markov Property:** A variable is conditionally independent of all other variables given its neighbors:
$$X_u \perp X_{V /N(u)}  | X_{N(u)} $$
Where $N(u)$ represents the set of neighbours of vertex $u$ according to $G=(V,E)$
3. **Global Markov Property:**  Any two subsets of variables are conditionally independent given a separating subset:
$$X_A \perp X_B  | X_S $$
Where every path from a node in $A$ to a node in $B$ passes through $S$.

One useful way of specifying the MRF is through the conecpt of clique potentials. A subgraph $C \subset G$ is a clique if it is a fully connected subgraph, the set of all cliques of $G$ is denoted: $Cl(G)$. We can then specify a joint distribution of random-variables $\mathbf{X} =  (X_v)_{v \in V}$ via clique potential functions: $\phi_C(X_C)$ such that:
$$P(\mathbf{X} = \mathbf{x}) = \prod_{C \in Cl(G)} \phi_C (x_C) $$
We note that every joint distribution specified in this was is a MRF. However the reverse is not necessarily true, we can however factorise a joint distribution this way if:
- The distribution is strictly positive (known as the Hammersley-Clifford theorem)
- The graph is chordal

A convenient form of clique potential is the exponential form which we can denote as:
$$ P(\mathbf{X} = \mathbf{x}) = \frac{1}{Z} exp\left(\sum_{C \in Cl(G)} w_C f_C(x_C)\right)$$
With:
$$ Z = \sum_{\mathbf{x}} exp\left(\sum_{C \in Cl(G)} w_C f_C(x_C)\right) $$
Which is nothing more than a general form of the Gibbs distribution we looked at in previous blog posts. To see this for the Ising model the cliques are adjacent pairs of spins $C_{xy} = (\sigma_x, \sigma_y)$ with $f_{C_{xy}} = \sigma_x\sigma_y$ and $w_{C_{xy}} = - J_{xy}$ - this is the simplest form of a MRF. We can then extend some of the ideas and results we have looked at previously to a much larger class of probabilistic models.

Markov random fields have been used in a variety of applications relating to computer vision and image/video processing including texture synthesis, compression, enhancement, etc. They have also been used in machine learning and computational biology. The ideas are also used in combinatorial optimization problems owing to their similarity to spin glasses.

## Hopfield Network
We now look at a(n artificial) neural-network model developed by John Hopfield in 1982. This model is no longer particularly in favour as it doesn't have particularly strong performance but is interesting nonetheless. We will look at Hopfield networks as a mechanism for pattern storage and recall, they can be used in different applications. The model took spin glass models as a direct inspiration.

A Hopfield network is a single layer recurrent network with a fully connected geometry, each neuron receives input from every other neuron and also gives output to every other neuron. We can specify the network through a weight matrix $(W_{ij})_{ij}$ which has the following properties:
- **Symmetry:** $W_{ij} = W_{ji} \; \; \; \forall i,j$
- **No self-connectivity:** $W_{ii} = 0 \; \; \; \forall i$

The neurons themselves take discrete values of $\sigma_i \pm 1$ denoting "on" or "off" states. At each update step we define a quantity:
$$ \theta_i = \sum_j W_{ij} \sigma_j + b_i$$
Where $b_i$ is a bias term (we can take $b_i = 0$). If $\theta_i > 0$ then we set $\sigma_i = 1$ and vice-versa. We can also note that this induces an energy function:
$$ E = - \frac{1}{2}\sum_{ij} W_{ij} \sigma_i \sigma_j $$
Which is exactly the Hamiltonian we investigated in our spin glass blog posts. We can see that the Hopfield Network is in fact nothing more than a Sherrington-Kirkpatrick spin glass viewed in a different way. From fairly basic arguments we can see that for each step of the dynamics described above the energy decreases.

Now suppose we want the network to "remember" patterns, that is we want it to remember a certain "on-off" configuration of neurons. To do this we have to select the correct weight matrix. By noting that the dynamics suggest each step leads to an energy decrease we want our stored patterns to be local energy minima of the network. Suppose we want to store an $N$ bit pattern: $\hat{x} = \left( \hat{x}_1, \hat{x}_2, ... , \hat{x}_N \right)$ if we take: $W_{ij} = c \hat{x}_i\hat{x}_j$ for some constant $c>0$ (representing the learning rate), then we have:
$$ \theta_i = \sum_j W_{ij} \sigma_j = \sum_j c \hat{x}_i\hat{x}_j \sigma_j = c\sum_j \hat{x}_i = c(N-1)\hat{x}_i$$
And so we have $\hat{x}$ forms a(n attractive) fixed point of the dynamics. Starting from a noisy input configuration eventually we should end up with the stored pattern.

We can naturally extend this further if we want to store $P$ different patterns $\left(\hat{x}^p\right)_{p=1}^P$:
$$W_{ij} = c \sum_p \hat{x}_i^p\hat{x}_j^p$$
Typically we would take $c$ to scale with the number of patterns, such as: $c=\frac{1}{P}$.

From basic arguments and intuition we can see that recall will deteriorate as we add more and more patterns. Can we construct an argument to find a "capacity" of the network? Yes! And it is quite simple. We start by re-writing the sum for $\theta_i(\hat{x}^p)$ (that is $\theta_i$ evaluated at one of the stored patterns $p$):
$$\theta_i(\hat{x}^p) = \sum_j W_{ij} \hat{x}^p_j = \frac{1}{P} \sum_j\sum_q \hat{x}_i^q\hat{x}_j^q\hat{x}^p_j = \hat{x}_i^p +   \frac{1}{P} \sum_j\sum_{q \neq p} \hat{x}_i^q\hat{x}_j^q\hat{x}^p_j  $$
Where the second term (the double summation) is referred to as the crosstalk term, if this is less than 1 then $\hat{x}^p$ is a fixed point and the network has "remembered" it. We can then define:
$$ C_i^p = - \hat{x}_i^p \frac{1}{P} \sum_j\sum_{q \neq p} \hat{x}_i^q\hat{x}_j^q\hat{x}^p_j $$
If $C_i^p<0$ then $\hat{x}^p$ will be a fixed point. For $C_i^p>1$ then there is instability. We want to find $P(C_i^p > 1)$. By considering the limit of a large number of neurons and patterns we can consider the case of the terms $\hat{x}_i^q\hat{x}_j^q$ being uniformly random. By applying the central limit theorem we get: $C_i^p \sim N(0,\sigma^2)$ with $\sigma^2 =\frac{P}{N}$ - that is the ratio of the number of stored patterns over the number of neurons. For example if we have $\frac{P}{N} = 0.37$ then the probability of observing an error is around $5%$. From further analysis (not shown here) we can find that these errors can "add up", for stability we require: $\frac{P}{N} < 0.138$. So for a $10x10$ neuron array we will only want to store 13 patterns or fewer.

So far we have assumed that each pattern is treated equally by the network, we can remember some patterns "more strongly" by assigning them a multiplicity (essentially the number of times the pattern is "remembered"). With higher multiplicity the pattern will have a larger basin of attraction. However we have to consider the sum of multiplicities now as opposed to the number of unique patterns. For example for the $10x10$ neuron case, if we have a pattern with multiplicity $10$ then we can only store a maximum of 3 extra patterns (of multiplicity 1).

So far the picture looks rosy with our Hopfield-Network, unfortunately there are a few issues that prevents them being widely used. Firstly we have the occurence of "spurious states" - these are non-remembered states that are attractors of the network dynamics. If we start near one of these spurious states the dynamics will converge towards them, which is obviously not ideal. These are due to the complex energy landscape of the Sherrington-Kirkpatrick spin-glass. One simple example of a spurious state would be any inverse image of a stored pattern (i.e. flipping +1 neuron activations to -1 and vice versa). There can be other spurious states however.

We now present a basic implementation of a Hopfield network as an illustration. We will consider a $7x7$ neuron grid and aim to store $4$ patterns. We will "flatten" the neuron grid to a $49$ element single dimension array for convenience.

```python
# A code implementing a Hopfield network
# Storing a number of 2d images for recall

import matplotlib.pyplot as plt
import numpy as np

# Set random seed for reproducibility
np.random.seed(123)

num_patterns = 4
grid_width = 7
grid_height = 7
grid_size = grid_width*grid_height

# Fix stored patterns
stored = np.zeros((num_patterns, grid_size))

stored[0] = [-1, -1, +1, +1, -1, -1, -1, \
             -1, +1, -1, +1, -1, -1, -1, \
             -1, -1, -1, +1, -1, -1, -1, \
             -1, -1, -1, +1, -1, -1, -1, \
             -1, -1, -1, +1, -1, -1, -1, \
             -1, -1, -1, +1, -1, -1, -1, \
             -1, +1, +1, +1, +1, +1, -1 ]
stored[1] = [-1, +1, +1, +1, +1, +1, +1, \
             +1, -1, -1, -1, -1, -1, +1, \
             -1, -1, -1, -1, -1, +1, +1, \
             -1, +1, +1, +1, +1, +1, -1, \
             +1, +1, -1, -1, -1, -1, -1, \
             +1, -1, -1, -1, -1, -1, -1, \
             +1, +1, +1, +1, +1, +1, +1 ]
stored[2] = [-1, +1, +1, +1, +1, +1, +1, \
             +1, -1, -1, -1, -1, -1, +1, \
             -1, -1, -1, -1, -1, -1, +1, \
             -1, -1, +1, +1, +1, +1, -1, \
             -1, -1, -1, -1, -1, -1, +1, \
             +1, -1, -1, -1, -1, -1, +1, \
             -1, +1, +1, +1, +1, +1, +1 ]
stored[3] = [-1, -1, -1, +1, +1, -1, -1, \
             -1, -1, +1, -1, +1, -1, -1, \
             -1, +1, -1, -1, +1, -1, -1, \
             +1, -1, -1, -1, +1, -1, -1, \
             +1, +1, +1, +1, +1, +1, +1, \
             -1, -1, -1, -1, +1, -1, -1, \
             -1, -1, -1, -1, +1, -1, -1 ]

# Display the patterns
fig, ax = plt.subplots(1, num_patterns, figsize=(10, 5))

for i in range(num_patterns):
    ax[i].imshow(stored[i].reshape((grid_height, grid_width)), cmap='binary')
    ax[i].set_xticks([])
    ax[i].set_yticks([])
    ax[i].set_title("Pattern %i" %(i+1))
fig.suptitle("Remembered Patterns", x=0.5, y=0.8, size='xx-large')
plt.show()

# Create network weights matrix
W = np.zeros((grid_size, grid_size))

for i in range(grid_size):
    for j in range(grid_size):
        if i == j or W[i, j] != 0.0:
            continue

        w = 0.0

        for n in range(num_patterns):
            w += stored[n, i] * stored[n, j]

        W[i, j] = w / num_patterns
        W[j, i] = W[i, j]

# Test noisy inputs
noise_rate = 0.25
num_cycles = 5

fig, ax = plt.subplots(num_patterns, 3, figsize=(10, 15))

for x in range(num_patterns):
    original_image = stored[x]
    noisy_image = original_image.copy()

    for i in range(grid_size):
        if np.random.random() < noise_rate:
            noisy_image[i] *= -1

    test_image = noisy_image.copy()

    for _ in range(num_cycles):
        for i in range(grid_size):
            test_image[i] = 1.0 if np.dot(W[i], test_image) > 0 else -1.0

    # Display results
    ax[x,0].imshow(noisy_image.reshape((grid_height, grid_width)), cmap='binary')
    ax[x,0].set_xticks([])
    ax[x,0].set_yticks([])
    ax[x,0].set_title("Noisy Input Pattern %i" %(x+1))
    ax[x,1].imshow(test_image.reshape((grid_height, grid_width)), cmap='binary')
    ax[x,1].set_xticks([])
    ax[x,1].set_yticks([])
    ax[x,1].set_title("Hopfield Network Prediction")
    ax[x,2].imshow(original_image.reshape((grid_height, grid_width)), cmap='binary')
    ax[x,2].set_xticks([])
    ax[x,2].set_yticks([])
    ax[x,2].set_title("Target Pattern %i" %(x+1))
fig.suptitle("Noisy Image Retrieval (Noise Rate=%d" %(noise_rate*100) +"$\%$)", x=0.5, y=0.91, size='xx-large')
plt.show()

# Calculate energies of stored patterns
energy = np.zeros(num_patterns)

for x in range(num_patterns):
    e = 0
    for i in range(grid_size):
        for j in range(grid_size):
            e += W[i,j] * stored[x][i]* stored[x][j]
    e *= -0.5
    print("Energy of pattern ",x+1,":",e)
```

In this toy example we have seen results that are not unreasonable, however even in this relatively simple example we see that patterns 2 and 3 are not recalled exactly - most likely as there is a high degree of overlap between them, each time the network dynamics converged to a spurious pattern.

Hopfield networks were the basis for the development of Boltzmann machines however which are stochastic versions of the Hopfield network. We will not touch on these here.

## NK Model
We now focus our attention on another model with a similarity to spin-glasses: the NK model propsed by Stuart Kauffmann. Described as a "tunable ruggedness" model by Kauffmann, the NK model originally proposed as a model of evolution (not to be confused with an "evolutionary algoirthm"). The model controls the size of a landscape and the number of local optima via the 2 parameters $N$ and $K$. The $N$ parameter denotes the degrees of freedom or the "size" of the system. the parameter $K$ represnts the level of interactivity between those degrees of freedom. $K$ varies from $K=0$ (leading to a smooth landscape) through to $K=N-1$ the most rugged/peaked.

In the context of spin-glass models we can express the Hamiltonians corresponding to the NK model as:
\begin{align}
K=0  \implies H &= - \sum_{i} J_{i} \sigma_{i} \\
K=1  \implies H &= - \sum_{i} J_{i} \sigma_{i} - \sum_{ij} J_{ij} \sigma_{i}\sigma_{j} \\
K=2  \implies H &= - \sum_{i} J_{i} \sigma_{i} - \sum_{ij} J_{ij} \sigma_{i}\sigma_{j} - \sum_{ijk} J_{ijk} \sigma_{i}\sigma_{j}\sigma_{k}
\end{align}
And so on. through this interpretation we can see that the NK model is nothing more than a generalization of the spin glass models we have looked at previously. However the model did not use spin-glasses as a basis, it was developed as a way of studying evolutionary systems and how one can navigate a fitness landscape.

As one would expect given the similarity the NK model shares some of the properties of spin-glasses; most notably that it has a very complicated landscape that often makes it hard to find even local optima. If the model sufficiently captures the properties of evolutionary systems this has some interesting implications.

We'll now present an NK model outside of the context of spin-glasses - namely as an evolution model as originally intended. We start by considering a sequence of genes $(s_i)_{i=1}^N$. We want to observe what happens to these genes over time. Within this model we assume an organism ($\mathbf{s}$) is completely defined by its gene sequence. The fitness of an organism is defined as an average of the fitness of its genes:
$$ F(\mathbf{s}) = \frac{1}{N} \sum_i f(s_i)$$
The fitness function depends on the value of the gene itself but also other genes - in biology this is known as "epistasis". We use the matrix $\left( A_{ij} \right)_{i,j=1}^N$ to denote dependence. If $A_{ij} = 1$ then gene $i$ depends on gene $j$ and is zero otherwise. Note that this matrix denotes a directed network and need not be symmetric. We can then compute a fitness landscape for each organism according to this equation.

In the NK model we assume that initially the entire population consists of one organism (sequence of genes). At random a mutation occurs to one of the genes, this new mutation either dies out quickly (if it has a lower fitness) or it reproduces faster than the original organism (which dies out) and becomes the sole organism in the system. The assumption here of course is that genetic mutations are far enough apart in time that the system can "settle" at each step.

Given a fitness landscape we can specify the dynamics of the NK model as:
1. From $\mathbf{s} = (s_1, ..., s_N)$ pick a gene $i \in {1, ... , N}$ at random and mutate its value to create a new organism $\mathbf{\hat{s}}$
2. If $F(\mathbf{\hat{s}}) > F(\mathbf{s})$ use $\mathbf{\hat{s}}$ as the new state, otherwise keep $\mathbf{s}$ and repeat

Clearly this is simply a greedy hill climber algorithm and the dynamics will end up in a nearby local maxima. The main attraction of the NK model is through the development of the fitness landscape.

The NK model has been developed further in the NKCS model. The NK model essentially looks at genetic variation of one species, the NKCS model extends this idea to multiple species. To do this we assume each species follows its own NK dynamics, its dynamics also interacts with $S$ other species. Each gene in a species is coupled to $C$ randomly chosen genes from the $S$ species. The dynamics can then be described as:
1.  Sequentially select each species during a time step
2. With the selected species mutate one of the genes
3. Calculate the updated fitness and either accept or reject this mutation and repeat

Whereas the NK model always ends up in a local fitness maxima this is not necessarily true of the NKCS model since for a fixed point we would require each species to be in a fitness maxima at the same time. This allows the model to display some more sophsticated dynamics such as criticality, self-organised-criticality and co-evolution (and others).

In addition to evolution these models can also be used as a model of immunity. Along with this NKCS models have been used as a model of technological evolution. The model is general enough that it can be applied to, essentially, any evolutionary system.

These models can get fairly complicated, in the future I may write a full length post on models of evolution.

## Conclusion
We have now seen that we can extend our spin-glass models to more sophisticated spin-glass models through the Potts model which are more physically plausible. We also saw that spin-glass models were a major influence on some of the first artificial neural networks. Lastly by looking at spin-glasses from a different point of view we have seen how they can inform our understanding of evolutionary systems.

We did not cover it here but spin-glasses have also influenced combinatorial optimization problems. We touched upon this in a previous post where we introduced the concept of simulated annealing. These techniques have been used in computer chip design (where one needs to optimize size/geometry of chip design versus performance). They have also been used in studying protein folding where there is a large universe of possible orientations that need to be searched according to a number of constraints.


# **Electronic structure**

## **Introduction**

The molecular Hamiltonian is

$$
\mathcal{H} = - \sum_I \frac{\nabla_{R_I}^2}{M_I} - \sum_i \frac{\nabla_{r_i}^2}{m_e} - \sum_I\sum_i  \frac{Z_I e^2}{|R_I-r_i|} + \sum_i \sum_{j>i} \frac{e^2}{|r_i-r_j|} + \sum_I\sum_{J>I} \frac{Z_I Z_J e^2}{|R_I-R_J|}
$$

Because the nuclei are much heavier than the electrons they do not move on the same time scale and therefore, the behavior of nuclei and electrons can be decoupled. This is the Born-Oppenheimer approximation.

Therefore, one can first tackle the electronic problem with nuclear coordinate entering only as parameters. The energy levels of the electrons in the molecule can be found by solving the non-relativistic time independent Schroedinger equation,

$$
\mathcal{H}_{\text{el}} |\Psi_{n}\rangle = E_{n} |\Psi_{n}\rangle
$$

where

$$
\mathcal{H}_{\text{el}} = - \sum_i \frac{\nabla_{r_i}^2}{m_e} - \sum_I\sum_i  \frac{Z_I e^2}{|R_I-r_i|} + \sum_i \sum_{j>i} \frac{e^2}{|r_i-r_j|}.
$$

In particular the ground state energy is given by:
$$
E_0 = \frac{\langle \Psi_0 | H_{\text{el}} | \Psi_0 \rangle}{\langle \Psi_0 | \Psi_0 \rangle}
$$
where $\Psi_0$ is the ground state of the system.

However, the dimensionality of this problem grows exponentially with the number of degrees of freedom. To tackle this issue we would like to prepare $\Psi_0$ on a quantum computer and measure the Hamiltonian expectation value (or $E_0$) directly.

So how do we do that concretely?

## **The Hartree-Fock initial state**

A good starting point for solving this problem is the Hartree-Fock (HF) method. This method approximates a N-body problem into N one-body problems where each electron evolves in the mean-field of the others. Classically solving the HF equations is efficient and leads to the exact exchange energy but does not include any electron correlation. Therefore, it is usually a good starting point to start adding correlation.

The Hamiltonian can then be re-expressed in the basis of the solutions of the HF method, also called Molecular Orbitals (MOs):

$$
\hat{H}_{elec}=\sum_{pq} h_{pq} \hat{a}^{\dagger}_p \hat{a}_q +
\frac{1}{2} \sum_{pqrs} h_{pqrs}  \hat{a}^{\dagger}_p \hat{a}^{\dagger}_q \hat{a}_r  \hat{a}_s
$$
with the 1-body integrals
$$
h_{pq} = \int \phi^*_p(r) \left( -\frac{1}{2} \nabla^2 - \sum_{I} \frac{Z_I}{R_I- r} \right)   \phi_q(r)dr
$$
and 2-body integrals
$$
h_{pqrs} = \int \frac{\phi^*_p(r_1)  \phi^*_q(r_2) \phi_r(r_2)  \phi_s(r_1)}{|r_1-r_2|}dr_1dr_2.
$$

The MOs ($\phi_u$) can be occupied or virtual (unoccupied). One MO can contain 2 electrons. However, in what follows we actually work with Spin Orbitals which are associated with a spin up ($\alpha$) of spin down ($\beta$) electron. Thus Spin Orbitals can contain one electron or be unoccupied.

We now show how to concretely realise these steps with Qiskit.

Qiskit is interfaced with different classical codes which are able to find the HF solutions. Interfacing between Qiskit and the following codes is already available:
* Gaussian
* Psi4
* PyQuante
* PySCF

In the following we set up a PySCF driver, for the hydrogen molecule at equilibrium bond length (0.735 angstrom) in the singlet state and with no charge.

```python
!pip install qiskit_nature
```

    Requirement already satisfied: qiskit_nature in /usr/local/lib/python3.7/dist-packages (0.3.0)
    Requirement already satisfied: qiskit-terra>=0.19.0 in /usr/local/lib/python3.7/dist-packages (from qiskit_nature) (0.19.1)
    Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.7/dist-packages (from qiskit_nature) (1.19.5)
    Requirement already satisfied: retworkx>=0.10.1 in /usr/local/lib/python3.7/dist-packages (from qiskit_nature) (0.10.2)
    Requirement already satisfied: psutil>=5 in /usr/local/lib/python3.7/dist-packages (from qiskit_nature) (5.4.8)
    Requirement already satisfied: setuptools>=40.1.0 in /usr/local/lib/python3.7/dist-packages (from qiskit_nature) (57.4.0)
    Requirement already satisfied: scikit-learn>=0.20.0 in /usr/local/lib/python3.7/dist-packages (from qiskit_nature) (1.0.2)
    Requirement already satisfied: h5py in /usr/local/lib/python3.7/dist-packages (from qiskit_nature) (3.1.0)
    Requirement already satisfied: scipy>=1.4 in /usr/local/lib/python3.7/dist-packages (from qiskit_nature) (1.7.3)
    Requirement already satisfied: tweedledum<2.0,>=1.1 in /usr/local/lib/python3.7/dist-packages (from qiskit-terra>=0.19.0->qiskit_nature) (1.1.1)
    Requirement already satisfied: python-constraint>=1.4 in /usr/local/lib/python3.7/dist-packages (from qiskit-terra>=0.19.0->qiskit_nature) (1.4.0)
    Requirement already satisfied: stevedore>=3.0.0 in /usr/local/lib/python3.7/dist-packages (from qiskit-terra>=0.19.0->qiskit_nature) (3.5.0)
    Requirement already satisfied: dill>=0.3 in /usr/local/lib/python3.7/dist-packages (from qiskit-terra>=0.19.0->qiskit_nature) (0.3.4)
    Requirement already satisfied: sympy>=1.3 in /usr/local/lib/python3.7/dist-packages (from qiskit-terra>=0.19.0->qiskit_nature) (1.7.1)
    Requirement already satisfied: symengine>=0.8 in /usr/local/lib/python3.7/dist-packages (from qiskit-terra>=0.19.0->qiskit_nature) (0.8.1)
    Requirement already satisfied: python-dateutil>=2.8.0 in /usr/local/lib/python3.7/dist-packages (from qiskit-terra>=0.19.0->qiskit_nature) (2.8.2)
    Requirement already satisfied: ply>=3.10 in /usr/local/lib/python3.7/dist-packages (from qiskit-terra>=0.19.0->qiskit_nature) (3.11)
    Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil>=2.8.0->qiskit-terra>=0.19.0->qiskit_nature) (1.15.0)
    Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from scikit-learn>=0.20.0->qiskit_nature) (3.0.0)
    Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.7/dist-packages (from scikit-learn>=0.20.0->qiskit_nature) (1.1.0)
    Requirement already satisfied: pbr!=2.1.0,>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from stevedore>=3.0.0->qiskit-terra>=0.19.0->qiskit_nature) (5.8.0)
    Requirement already satisfied: importlib-metadata>=1.7.0 in /usr/local/lib/python3.7/dist-packages (from stevedore>=3.0.0->qiskit-terra>=0.19.0->qiskit_nature) (4.10.0)
    Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata>=1.7.0->stevedore>=3.0.0->qiskit-terra>=0.19.0->qiskit_nature) (3.7.0)
    Requirement already satisfied: typing-extensions>=3.6.4 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata>=1.7.0->stevedore>=3.0.0->qiskit-terra>=0.19.0->qiskit_nature) (3.10.0.2)
    Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.7/dist-packages (from sympy>=1.3->qiskit-terra>=0.19.0->qiskit_nature) (1.2.1)
    Requirement already satisfied: cached-property in /usr/local/lib/python3.7/dist-packages (from h5py->qiskit_nature) (1.5.2)

```python
!pip install pyscf
```

    Collecting pyscf
      Downloading pyscf-2.0.1-cp37-cp37m-manylinux1_x86_64.whl (37.5 MB)
         |████████████████████████████████| 37.5 MB 1.1 MB/s
    Requirement already satisfied: numpy!=1.16,!=1.17,>=1.13 in /usr/local/lib/python3.7/dist-packages (from pyscf) (1.19.5)
    Requirement already satisfied: h5py>=2.7 in /usr/local/lib/python3.7/dist-packages (from pyscf) (3.1.0)
    Requirement already satisfied: scipy!=1.5.0,!=1.5.1 in /usr/local/lib/python3.7/dist-packages (from pyscf) (1.7.3)
    Requirement already satisfied: cached-property in /usr/local/lib/python3.7/dist-packages (from h5py>=2.7->pyscf) (1.5.2)
    Installing collected packages: pyscf
    Successfully installed pyscf-2.0.1

```python
from qiskit_nature.drivers import UnitsType, Molecule
from qiskit_nature.drivers.second_quantization import (
    ElectronicStructureDriverType,
    ElectronicStructureMoleculeDriver,
)

molecule = Molecule(
    geometry=[["H", [0.0, 0.0, 0.0]], ["H", [0.0, 0.0, 0.735]]], charge=0, multiplicity=1
)
driver = ElectronicStructureMoleculeDriver(
    molecule, basis="sto3g", driver_type=ElectronicStructureDriverType.PYSCF
)
```

For further information about the drivers see https://qiskit.org/documentation/nature/apidocs/qiskit_nature.drivers.second_quantization.html

## **The mapping from fermions to qubits**

The Hamiltonian given in the previous section is expressed in terms of fermionic operators. To encode the problem into the state of a quantum computer, these operators must be mapped to spin operators (indeed the qubits follow spin statistics).

There exist different mapping types with different properties. Qiskit already supports the following mappings:
* The Jordan-Wigner 'jordan_wigner' mapping (über das paulische äquivalenzverbot. In The Collected Works of Eugene Paul Wigner (pp. 109-129). Springer, Berlin, Heidelberg (1993)).
* The Parity 'parity' (The Journal of chemical physics, 137(22), 224109 (2012))
* The Bravyi-Kitaev 'bravyi_kitaev' (Annals of Physics, 298(1), 210-226 (2002))

The Jordan-Wigner mapping is particularly interesting as it maps each Spin Orbital to a qubit (as shown on the Figure above).

Here we set up the Electronic Structure Problem to generate the Second quantized operator and a qubit converter that will map it to a qubit operator.

```python
from qiskit_nature.problems.second_quantization import ElectronicStructureProblem
from qiskit_nature.converters.second_quantization import QubitConverter
from qiskit_nature.mappers.second_quantization import JordanWignerMapper, ParityMapper
```

```python
es_problem = ElectronicStructureProblem(driver)
second_q_op = es_problem.second_q_ops()
print(second_q_op[0])
```

    Fermionic Operator
    register length=4, number terms=14
      (0.18093119978423106+0j) * ( +_0 -_1 +_2 -_3 )
    + (-0.18093119978423128+0j) * ( +_0 -_1 -_2 +_3 )
    + (-0.18093119978423128+0j) * ( -_0 +_1 +_2 -_3 )
    + (0.18093119978423144+0j) * ( -_0 +_1 -_2 +_3  ...

If we now transform this Hamiltonian for the given driver defined above we get our qubit operator:

```python
qubit_converter = QubitConverter(mapper=JordanWignerMapper())
qubit_op = qubit_converter.convert(second_q_op[0])
print(qubit_op)
```

    -0.8105479805373281 * IIII
    - 0.22575349222402358 * ZIII
    + 0.17218393261915543 * IZII
    + 0.12091263261776633 * ZZII
    - 0.22575349222402358 * IIZI
    + 0.17464343068300436 * ZIZI
    + 0.16614543256382414 * IZZI
    + 0.17218393261915543 * IIIZ
    + 0.16614543256382414 * ZIIZ
    + 0.16892753870087904 * IZIZ
    + 0.12091263261776633 * IIZZ
    + 0.04523279994605781 * XXXX
    + 0.04523279994605781 * YYXX
    + 0.04523279994605781 * XXYY
    + 0.04523279994605781 * YYYY

In the minimal (STO-3G) basis set 4 qubits are required. We can reduce the number of qubits by using the Parity mapping, which allows for the removal of 2 qubits by exploiting known symmetries arising from the mapping.

```python
qubit_converter = QubitConverter(mapper=ParityMapper(), two_qubit_reduction=True)
qubit_op = qubit_converter.convert(second_q_op[0], num_particles=es_problem.num_particles)
print(qubit_op)
```

    -1.0523732457728605 * II
    + (-0.39793742484317884+1.3877787807814457e-17j) * ZI
    + (0.39793742484317884-2.7755575615628914e-17j) * IZ
    + (-0.011280104256235116+1.3877787807814457e-17j) * ZZ
    + (0.18093119978423114-3.469446951953614e-18j) * XX

This time only 2 qubits are needed.

Now that the Hamiltonian is ready, it can be used in a quantum algorithm to find information about the electronic structure of the corresponding molecule.

```python
import qiskit.tools.jupyter

%qiskit_version_table
```

Version Information

Qiskit Software	Version

qiskit-terra	0.19.1

qiskit-nature	0.3.0

System information

Python version	3.7.12

Python compiler	GCC 7.5.0

Python build	default, Sep 10 2021 00:21:48

OS	Linux

CPUs	1

Memory (Gb)	12.686653137207031

Tue Jan 18 02:00:56 2022 UTC


```
%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np
from sympy import symbols, eye, collect
from sympy.matrices import Matrix

from math import sqrt
%load_ext cythonmagic
```

```
b1, b2, b3 = symbols('b1, b2, b3', real=True)
a11, a12, a13, a21, a22, a23, a31, a32, a33 = symbols('a11, a12, a13, a21, a22, a23, a31, a32, a33', real=True)
L = symbols('L', real=True)

b = Matrix([b1, b2, b3])
A = Matrix([[a11, a12, a13],
            [a21, a22, a23],
            [a31, a32, a33]])

C = L * eye(3) + (1 - L) * A

p = collect((L * (1 - L) * b.T * C.adjugate() * b)[0].expand(), L)
p_prime = p.diff(L)

q = collect(C.det(), L)
q_prime = q.diff(L)

f = p / q

h = collect((p_prime * q - p * q_prime).expand(), L)
```

```
%%file parametric_function_roots.py
from sympy import symbols, eye, collect
from sympy.matrices import Matrix
from sympy.utilities.codegen import codegen

b1, b2 = symbols('b1, b2', real=True)
a11, a12, a21, a22 = symbols('a11, a12, a21, a22', real=True)
L = symbols('L', real=True)

b = Matrix([b1, b2])
A = Matrix([[a11, a12],
            [a21, a22]])

C = L * eye(2) + (1 - L) * A

p = collect((L * (1 - L) * b.T * C.adjugate() * b)[0].expand(), L)
p_prime = p.diff(L)

q = collect(C.det(), L)
q_prime = q.diff(L)

h = collect((p_prime * q - p * q_prime).expand(), L)

f = p / q

h0 = h.coeff(L, 0)
h1 = h.coeff(L, 1)
h2 = h.coeff(L, 2)
h3 = h.coeff(L, 3)
h4 = h.coeff(L, 4)

codegen([("f", f), ("h0", h0), ("h1", h1), ("h2", h2), ("h3", h3), ("h4", h4)], "C", "parametric_function_roots", to_files=True, header=False, empty=True)
```

    Overwriting parametric_function_roots.py

```
def p_AB(A, b, L):
    a11 = A[0, 0]
    a12 = A[0, 1]
    a21 = A[1, 0]
    a22 = A[1,1]
    b1 = b[0, 0]
    b2 = b[1, 0]

    return L**3*(a11*b2**2 - a12*b1*b2 - a21*b1*b2 + a22*b1**2 - b1**2 - b2**2) + L**2*(-2*a11*b2**2 + 2*a12*b1*b2 + 2*a21*b1*b2 - 2*a22*b1**2 + b1**2 + b2**2) + L*(a11*b2**2 - a12*b1*b2 - a21*b1*b2 + a22*b1**2)

def q_AB(A, b, L):
    a11 = A[0, 0]
    a12 = A[0, 1]
    a21 = A[1, 0]
    a22 = A[1,1]
    b1 = b[0, 0]
    b2 = b[1, 0]

    return L**2*(a11*a22 - a11 - a12*a21 - a22 + 1) + L*(-2*a11*a22 + a11 + 2*a12*a21 + a22) + a11*a22 - a12*a21

def f_AB(A, b, L):
    return p_AB(A, b, L) / q_AB(A, b, L)

def h_coeffs(A, b):
    a11 = A[0, 0]
    a12 = A[0, 1]
    a21 = A[1, 0]
    a22 = A[1,1]
    b1 = b[0, 0]
    b2 = b[1, 0]

    coeff = [a11**2*a22*b2**2 - a11**2*b2**2 - a11*a12*a21*b2**2 - a11*a12*a22*b1*b2 + a11*a12*b1*b2 - a11*a21*a22*b1*b2 + a11*a21*b1*b2 + a11*a22**2*b1**2 - 2*a11*a22*b1**2 - 2*a11*a22*b2**2 + a11*b1**2 + 2*a11*b2**2 + a12**2*a21*b1*b2 + a12*a21**2*b1*b2 - a12*a21*a22*b1**2 + a12*a21*b1**2 + a12*a21*b2**2 + a12*a22*b1*b2 - a12*b1*b2 + a21*a22*b1*b2 - a21*b1*b2 - a22**2*b1**2 + 2*a22*b1**2 + a22*b2**2 - b1**2 - b2**2,
             -4*a11**2*a22*b2**2 + 2*a11**2*b2**2 + 4*a11*a12*a21*b2**2 + 4*a11*a12*a22*b1*b2 - 2*a11*a12*b1*b2 + 4*a11*a21*a22*b1*b2 - 2*a11*a21*b1*b2 - 4*a11*a22**2*b1**2 + 6*a11*a22*b1**2 + 6*a11*a22*b2**2 - 2*a11*b1**2 - 2*a11*b2**2 - 4*a12**2*a21*b1*b2 - 4*a12*a21**2*b1*b2 + 4*a12*a21*a22*b1**2 - 4*a12*a21*b1**2 - 4*a12*a21*b2**2 - 2*a12*a22*b1*b2 - 2*a21*a22*b1*b2 + 2*a22**2*b1**2 - 2*a22*b1**2 - 2*a22*b2**2,
             6*a11**2*a22*b2**2 - a11**2*b2**2 - 6*a11*a12*a21*b2**2 - 6*a11*a12*a22*b1*b2 + a11*a12*b1*b2 - 6*a11*a21*a22*b1*b2 + a11*a21*b1*b2 + 6*a11*a22**2*b1**2 - 6*a11*a22*b1**2 - 6*a11*a22*b2**2 + a11*b1**2 + 6*a12**2*a21*b1*b2 + 6*a12*a21**2*b1*b2 - 6*a12*a21*a22*b1**2 + 5*a12*a21*b1**2 + 5*a12*a21*b2**2 + a12*a22*b1*b2 + a12*b1*b2 + a21*a22*b1*b2 + a21*b1*b2 - a22**2*b1**2 + a22*b2**2,
             -4*a11**2*a22*b2**2 + 4*a11*a12*a21*b2**2 + 4*a11*a12*a22*b1*b2 + 4*a11*a21*a22*b1*b2 - 4*a11*a22**2*b1**2 + 2*a11*a22*b1**2 + 2*a11*a22*b2**2 - 4*a12**2*a21*b1*b2 - 4*a12*a21**2*b1*b2 + 4*a12*a21*a22*b1**2 - 2*a12*a21*b1**2 - 2*a12*a21*b2**2,
             a11**2*a22*b2**2 - a11*a12*a21*b2**2 - a11*a12*a22*b1*b2 - a11*a21*a22*b1*b2 + a11*a22**2*b1**2 + a12**2*a21*b1*b2 + a12*a21**2*b1*b2 - a12*a21*a22*b1**2]
    return coeff

def ellipse_overlap_py(rA, radiiA, phiA, rB, radiiB, phiB):

    """

    X = Q^T * O^-2 * Q, where Q is rotation matrix given by angle phi and
    O is diagonal matrix with radii values along main diagonal

    What's needed is XA^-1 = Q^T * O^2 * Q

    """

    O = np.diag(radiiA ** 2)

    Q = np.array([[np.cos(phiA), np.sin(phiA)],
              [-np.sin(phiA), np.cos(phiA)]])

    XA = Q.T.dot(O).dot(Q)

    print str(XA)

    """

    X = Q^T * O^-2 * Q, where Q is rotation matrix given by angle phi and
    O is diagonal matrix with radii values along main diagonal

    What's needed is XB^1/2 = Q^T * O^-1 * Q

    """

    O = np.diag(radiiB ** -1)

    Q = np.array([[np.cos(phiB), np.sin(phiB)],
              [-np.sin(phiB), np.cos(phiB)]])

    XB = Q.T.dot(O).dot(Q)

    # Find rAB
    rAB = rB - rA

    # Find A_AB
    A_AB = XB.dot(XA).dot(XB)

    # Find a_AB
    a_AB = XB.dot(rAB)

    coeffs = h_coeffs(A_AB, a_AB)
    roots = np.roots(coeffs)

    F = None
    for r in roots:
        if np.isreal(r) and (0. <= r <= 1.):
            print 'max in (0, 1)'
            F =  f_AB(A_AB, a_AB, np.real(r))

    if F is None:
        if f_AB(A_AB, a_AB, 0) > f_AB(A_AB, a_AB, 1):
            print 'max at lambda = 0'
            F = f_AB(A_AB, a_AB, 0)
        else:
            print 'max at lambda = 1'
            F = f_AB(A_AB, a_AB, 1)

    return F, coeffs
```

```
# Ellipse A
rA = np.array([[0.], [0.]])
radiiA = np.array([1., 1.])
phiA = 0.

# Ellipse B
rB = np.array([[2.], [0.]])
radiiB = np.array([1.1, 1.])
phiB = 0.

F_c = ellipse_overlap_c(rA, radiiA, phiA, rB, radiiB, phiB)
F_py, coeffs = ellipse_overlap_py(rA, radiiA, phiA, rB, radiiB, phiB)

print F_py
print F_c
```

    [[ 1.  0.]
     [ 0.  1.]]
    max in (0, 1)
    0.907029478458
    0.907029478458

```
%%file matrix_operations.c

////////////////////////////////////////////////////////////////////////////////
//  void matrix_multiply(double *C, double *A, double *B, int nrows,          //
//                                                  int ncols, int mcols)     //
//                                                                            //
//  Description:                                                              //
//     Post multiply the nrows x ncols matrix A by the ncols x mcols matrix   //
//     B to form the nrows x mcols matrix C, i.e. C = A B.                    //
//     The matrix C should be declared as double C[nrows][mcols] in the       //
//     calling routine.  The memory allocated to C should not include any     //
//     memory allocated to A or B.                                            //
//                                                                            //
//  Arguments:                                                                //
//     double *C    Pointer to the first element of the matrix C.             //
//     double *A    Pointer to the first element of the matrix A.             //
//     double *B    Pointer to the first element of the matrix B.             //
//     int    nrows The number of rows of the matrices A and C.               //
//     int    ncols The number of columns of the matrices A and the           //
//                   number of rows of the matrix B.                          //
//     int    mcols The number of columns of the matrices B and C.            //
//                                                                            //
//  Return Values:                                                            //
//     void                                                                   //
//                                                                            //
//  Example:                                                                  //
//     #define N                                                              //
//     #define M                                                              //
//     #define NB                                                             //
//     double A[M][N],  B[N][NB], C[M][NB];                                   //
//                                                                            //
//     (your code to initialize the matrices A and B)                         //
//                                                                            //
//     matrix_multiply(&C[0][0], &A[0][0], &B[0][0], M, N, NB);               //
//     printf("The matrix C is \n"); ...                                      //
//                                                                            //
//  Source: http://www.mymathlib.com/matrices/arithmetic/mul_mat.html         //
//  Last visisted on: 2013-12-05                                              //
////////////////////////////////////////////////////////////////////////////////
void matrix_multiply(double *C, double *A, double *B, int nrows, int ncols, int mcols)
{
   double *pB;
   double *p_B;
   int i,j,k;

   for (i = 0; i < nrows; A += ncols, i++)
      for (p_B = B, j = 0; j < mcols; C++, p_B++, j++) {
         pB = p_B;
         *C = 0.0;
         for (k = 0; k < ncols; pB += mcols, k++)
            *C += *(A+k) * *pB;
      }
}

////////////////////////////////////////////////////////////////////////////////
//  void matrix_transpose(double *At, double *A, int nrows, int ncols)        //
//                                                                            //
//  Description:                                                              //
//     Take the transpose of A and store in At, i.e. At = A'.                 //
//     The matrix At should be declared as double At[ncols][nrows] in the     //
//     calling routine, and the matrix A declared as double A[nrows[ncols].   //
//     In general, At and A should be disjoint i.e. their memory locations    //
//     should be distinct.                                                    //
//                                                                            //
//  Arguments:                                                                //
//     double *At   Pointer to the first element of the matrix At.            //
//     double *A    Pointer to the first element of the matrix A.             //
//     int    nrows The number of rows of the matrix A and number of columns  //
//                  of the matrix At.                                         //
//     int    ncols The number of columns of the matrix A and the number of   //
//                  rows of the matrix At.                                    //
//                                                                            //
//  Return Values:                                                            //
//     void                                                                   //
//                                                                            //
//  Example:                                                                  //
//     #define N                                                              //
//     #define M                                                              //
//     double A[M][N],  At[N][M];                                             //
//                                                                            //
//     (your code to initialize the matrix A)                                 //
//                                                                            //
//     matrix_transpose(&At[0][0], &A[0][0], M, N);                           //
//     printf("The transpose of A is the matrix At \n"); ...                  //
//                                                                            //
//  http://www.mymathlib.com/c_source/matrices/datamovement/transpose_matrix.c//
//  Last visisted on: 2013-12-05                                              //
////////////////////////////////////////////////////////////////////////////////
void matrix_transpose(double *At, double *A, int nrows, int ncols)
{
   double *pA;
   double *pAt;
   int i,j;

   for (i = 0; i < nrows; At += 1, A += ncols, i++) {
      pAt = At;
      pA = A;
      for (j = 0; j < ncols; pAt += nrows, j++) *pAt = pA[j];
   }
}

////////////////////////////////////////////////////////////////////////////////
//  void set_diagonal(double *A, double v[], int nrows, int ncols)            //
//                                                                            //
//  Description:                                                              //
//     Copy the vector v to the diagonal of the matrix A, i.e.                //
//     A[i][i] = v[i], where 0 <= i < min( nrows, ncols ).                    //
//     Note that v should be declared "double v[N]", N >= min( nrows, ncols ) //
//     in the calling routine.                                                //
//                                                                            //
//  Arguments:                                                                //
//     double *A    Pointer to the first element of the destination matrix A. //
//     double v[]   Source of the new diagonal for the matrix A.              //
//     int    nrows The number of rows matrix A.                              //
//     int    ncols The number of columns of the matrix A.                    //
//                                                                            //
//  Return Values:                                                            //
//     void                                                                   //
//                                                                            //
//  Example:                                                                  //
//     #define N                                                              //
//     #define M                                                              //
//     double A[M][N],  v[N];                                                 //
//                                                                            //
//     (your code to initialize the matrix A and the vector v)                //
//                                                                            //
//     set_diagonal(&A[0][0], v, M, N);                                       //
//     printf("The matrix A is \n"); ...                                      //
//                                                                            //
//  http://www.mymathlib.com/c_source/matrices/datamovement/set_diagonal.c    //
//  Last visisted on: 2013-12-05                                              //
////////////////////////////////////////////////////////////////////////////////
void set_diagonal(double *A, double v[], int nrows, int ncols)
{
   int n;

   if (nrows < ncols) n = nrows; else n = ncols;

   for (; n > 0 ; A += (ncols + 1), n--)  *A = *v++;
}
```

    Overwriting matrix_operations.c

```
%%file matrix_operations.h
void matrix_multiply(double *C, double *A, double *B, int nrows, int ncols, int mcols);
void matrix_transpose(double *At, double *A, int nrows, int ncols);
void set_diagonal(double *A, double v[], int nrows, int ncols);
```

    Writing matrix_operations.h

```
%%file ellipse_overlap.c
#include <math.h>
#include <gsl/gsl_poly.h>
#include "matrix_operations.c"
#include "parametric_function_roots.c"
#include <stdio.h>

size_t remove_leading_zeros(double *h, size_t n)
{

    size_t m = n;
    double small = 1e-5;
    size_t i;
    for(i=n; i>0; i--)
    {

        if (fabs(h[i-1]) > small)
        {
            m = i;
            break;
        }
    }

    return m;

}

size_t find_roots(double *z, double *h, size_t n)
{
    size_t m;
    m = remove_leading_zeros(&h[0], n);

    gsl_poly_complex_workspace *w = gsl_poly_complex_workspace_alloc(m);
    gsl_poly_complex_solve(h, m, w, z);
    gsl_poly_complex_workspace_free (w);

    return m;

}

void characteristic_ellipse_matrix(double *X, double *R, double phi, double exponent)
{
    // rotation matrix
    double Q[2][2] = {{ 0 }};
    double Qt[2][2] = {{ 0 }};
    Q[0][0] = cos(phi);
    Q[0][1] = sin(phi);
    Q[1][0] = -sin(phi);
    Q[1][1] = cos(phi);
    matrix_transpose(&Qt[0][0], &Q[0][0], 2, 2);

    // radii matrix
    double O[2][2];
    R[0] = pow(R[0], exponent * -2.0);
    R[1] = pow(R[1], exponent * -2.0);
    set_diagonal(&O[0][0], R, 2, 2);

    // characteristic ellipse matrix
    double X_temp[2][2] = {{ 0 }};
    matrix_multiply(&X_temp[0][0], &O[0][0], &Q[0][0], 2, 2, 2);
    matrix_multiply(X, &Qt[0][0], &X_temp[0][0], 2, 2, 2);
}

double overlap_potential(double *rA, double *radiiA, double phiA, double *rB, double *radiiB, double phiB)
{
    // find XA^(-1) and XB^(1/2)
    double XA[2][2] = {{ 0 }};
    double XB[2][2] = {{ 0 }};
    characteristic_ellipse_matrix(&XA[0][0], &radiiA[0], phiA, -1.0);
    characteristic_ellipse_matrix(&XB[0][0], &radiiB[0], phiB, 0.5);

    // find A_AB
    double A_AB[2][2] = {{ 0 }};
    double A_temp[2][2] = {{ 0 }};
    matrix_multiply(&A_temp[0][0], &XA[0][0], &XB[0][0], 2, 2, 2);
    matrix_multiply(&A_AB[0][0], &XB[0][0], &A_temp[0][0], 2, 2, 2);

    // find r_AB
    double rAB[2];
    rAB[0] = rB[0] - rA[0];
    rAB[1] = rB[1] - rA[1];

    // find a_AB
    double a_AB[2];
    matrix_multiply(&a_AB[0], &XB[0][0], &rAB[0], 2, 2, 1);

    // extract elements of the matrix A_AB and a_AB
    double a11, a12, a21, a22;
    double b1, b2;
    a11 = A_AB[0][0];
    a12 = A_AB[0][1];
    a21 = A_AB[1][0];
    a22 = A_AB[1][1];
    b1 = a_AB[0];
    b2 = a_AB[1];

    FILE * fp_AB;
    fp_AB = fopen ("file-AB.txt", "w+");
    fprintf(fp_AB, "a11 = %f.\n", a11);
    fprintf(fp_AB, "a12 = %f.\n", a12);
    fprintf(fp_AB, "a21 = %f.\n", a21);
    fprintf(fp_AB, "a22 = %f.\n", a22);
    fprintf(fp_AB, "b1 = %f.\n", b1);
    fprintf(fp_AB, "b2 = %f.\n", b2);
    fclose(fp_AB);

    // find coefficients for the parametric polynomial derivative used to find max
    double h[5];
    double z[8] = { 0, 0, 0, 0, 0, 0, 0, 0 };
    size_t n = 5;
    h[0] = h0(a11, a12, a21, a22, b1, b2);
    h[1] = h1(a11, a12, a21, a22, b1, b2);
    h[2] = h2(a11, a12, a21, a22, b1, b2);
    h[3] = h3(a11, a12, a21, a22, b1, b2);
    h[4] = h4(a11, a12, a21, a22, b1, b2);

    // find roots
    size_t m;
    m = find_roots(&z[0], &h[0], n);

    double F;
    if (f(0, a11, a12, a21, a22, b1, b2) > f(1, a11, a12, a21, a22, b1, b2))
    {
       F = f(0, a11, a12, a21, a22, b1, b2);

    }
    else
    {
       F = f(1, a11, a12, a21, a22, b1, b2);
    }

    int i;
    for(i=0; i<=m-1; i++)
    {

       if (( z[2 * i + 1] == 0 ) & (z[2 * i] > 0) & (z[2 * i] < 1))
       {
          F = f(z[2 * i], a11, a12, a21, a22, b1, b2);
       }
    }

    return F;

}
```

    Overwriting ellipse_overlap.c

```cython
%%cython -I/home/adam/sync/IPyNotebooks -f -lm -lgsl -lgslcblas

import numpy as np
cimport numpy as np

cdef extern from "ellipse_overlap.c":
    double overlap_potential(double *rA, double *radiiA, double phiA, double *rB, double *radiiB, double phiB)
    size_t find_roots(double *z, double *h, size_t n)
    size_t remove_leading_zeros(double *h, size_t n)

def ellipse_overlap_c(r1, radii1, phi1, r2, radii2, phi2):
    cdef np.ndarray[double, ndim=1, mode="c"] rA = np.ascontiguousarray(r1.flatten())
    cdef np.ndarray[double, ndim=1, mode="c"] radiiA = np.ascontiguousarray(radii1.flatten())
    cdef double phiA = float(phi1)

    cdef np.ndarray[double, ndim=1, mode="c"] rB = np.ascontiguousarray(r2.flatten())
    cdef np.ndarray[double, ndim=1, mode="c"] radiiB = np.ascontiguousarray(radii2.flatten())
    cdef double phiB = float(phi2)

    cdef double F

    F = overlap_potential(&rA[0], &radiiA[0], phiA, &rB[0], &radiiB[0], phiB)

    return F

def poly_zeros(clist):
    cdef size_t n = len(clist)
    cdef size_t m
    cdef np.ndarray[double, ndim=1, mode="c"] z = np.ascontiguousarray(np.zeros(2*(n-1)), dtype=np.float64)
    cdef np.ndarray[double, ndim=1, mode="c"] h = np.ascontiguousarray(np.asarray(clist), dtype=np.float64)

    m = find_roots(&z[0], &h[0], n)

    return z[: 2*(m-1)]

def filter_zeros(clist):
    cdef size_t n = len(clist)
    cdef np.ndarray[double, ndim=1, mode="c"] h = np.ascontiguousarray(np.asarray(clist), dtype=np.float64)

    m = remove_leading_zeros(&h[0], n)

    return m

```

```
#clist = np.asarray([5.0, 4.0, 2.0, 3.0, 0.0])
clist = np.asarray([ 0.25, -0.5,  -0.75, 0.0 ])
clist = np.asarray([ 0.25, -0.5,  -0.75, 0.0 ])
m = filter_zeros(clist)
print m
```

    3

```
#clist = np.asarray([4.0, -8.0, 0.0, 0.0, 6.94472898422e-310])
#clist = np.asarray([4.0, -8.0, 0.0, 0.0, 0.0])
clist = np.asarray(np.random.rand(5))
#clist = np.asarray([1.0, 0.0, 0., 1.0])
out = poly_zeros(clist)
print out
print np.roots(np.flipud(clist))
```

    [-0.65214302  0.         -0.21432048  0.          0.3814299   1.50568235
      0.3814299  -1.50568235]
    [ 0.38142990+1.50568235j  0.38142990-1.50568235j -0.65214302+0.j
     -0.21432048+0.j        ]


# ¿Qué tan importante son las fuentes de ingreso para un aplicante de préstamos en línea?

```python
import pandas as pd
import numpy  as np
from scipy.stats import chi2_contingency
import statsmodels.api as sm
from sklearn.metrics import roc_curve, auc
from sklearn.model_selection import StratifiedKFold
import matplotlib.pyplot as plt
from scipy import interp
from statsmodels.formula.api import ols
import seaborn as sns
from matplotlib.widgets import Slider, Button, RadioButtons
from scipy.optimize import fsolve

#get_ipython().magic(u'matplotlib inline')
```

## Introducción (5 mts)

**Contexto de negocios.** Una compañía P2P tiene un sistema online que acepta o rechaza préstamos en línea. Se rechazan todos aquellos que no posean una fuente verificable de ingresos

**Problema.** ¿Debería de verificarse la fuente de ingresos antes de aprobar un préstamo?

**Contexto analítico.** fuente: [LendingClub Statistics](https://www.lendingclub.com/info/download-data.action) préstamos de 2007 - 2012. usaremos **regresión logística**, para responder la pregunta.

## Exploración de los datos (45 mts)

```python
df = pd.read_csv("Lending_club_cleaned_2.csv")
df.loan_status = df.loan_status.astype(pd.api.types.CategoricalDtype(categories=["Charged Off", "Fully Paid"]))
df.verification_status = df.verification_status.astype(pd.api.types.CategoricalDtype(categories=['Not Verified', 'Source Verified', 'Verified']))
df.emp_length = df.emp_length.astype(pd.api.types.CategoricalDtype(categories=["< 1 year", "1 year", "2 years", "3 years", "4 years", \
                                                             "5 years", "6 years", "7 years", "8 years", "9 years", \
                                                             "10+ years"]))
df.home_ownership = df.home_ownership.astype(pd.api.types.CategoricalDtype(categories=["RENT", "MORTGAGE","OWN","OTHER"]))
df.term = df.term.astype(pd.api.types.CategoricalDtype(categories=[" 36 months", " 60 months"]))
df.grade = df.grade.astype(pd.api.types.CategoricalDtype(categories=["A","B","C","D","E","F","G"]))
df.int_rate = df.int_rate.str.rstrip("%").astype("float")
```

```python
df.shape
```

    (38705, 10)

```python
df.head(10)
```

      loan_status
      annual_inc
      verification_status
      emp_length
      home_ownership
      int_rate
      loan_amnt
      purpose
      term
      grade

      0
      Fully Paid
      24000.0
      Verified
      10+ years
      RENT
      10.65
      5000
      credit_card
      36 months
      B

      1
      Charged Off
      30000.0
      Source Verified
      < 1 year
      RENT
      15.27
      2500
      car
      60 months
      C

      2
      Fully Paid
      12252.0
      Not Verified
      10+ years
      RENT
      15.96
      2400
      small_business
      36 months
      C

      3
      Fully Paid
      49200.0
      Source Verified
      10+ years
      RENT
      13.49
      10000
      other
      36 months
      C

      4
      Fully Paid
      80000.0
      Source Verified
      1 year
      RENT
      12.69
      3000
      other
      60 months
      B

      5
      Fully Paid
      36000.0
      Source Verified
      3 years
      RENT
      7.90
      5000
      wedding
      36 months
      A

      6
      Fully Paid
      47004.0
      Not Verified
      8 years
      RENT
      15.96
      7000
      debt_consolidation
      60 months
      C

      7
      Fully Paid
      48000.0
      Source Verified
      9 years
      RENT
      18.64
      3000
      car
      36 months
      E

      8
      Charged Off
      40000.0
      Source Verified
      4 years
      OWN
      21.28
      5600
      small_business
      60 months
      F

      9
      Charged Off
      15000.0
      Verified
      < 1 year
      RENT
      12.69
      5375
      other
      60 months
      B

38706 observaciones. Cada una corresponde a un crédito aprobado. La primera columna indica si fue pagado (Fully paid) o no (Charged Off). Las otras 9 columnas son las siguientes:

|      annual_inc     |                                                 The self-reported annual income provided by the borrower during registration.                                                |
|:-------------------:|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|
| verification_status |                                          Indicates if income was verified by LC, not verified, or if the income source was verified                                          |
|      emp_length     |                       Employment length in years. Possible values are between 0 and 10 where 0 means less than one year and 10 means ten or more years.                      |
|    home_ownership   |             The home ownership status provided by the borrower during registration or obtained from the credit report. Our values are: RENT, OWN, MORTGAGE, OTHER            |
|       int_rate      |                                                                           Interest Rate on the loan                                                                          |
|      loan_amnt      | The listed amount of the loan applied for by the borrower. If at some point in time, the credit department reduces the loan amount, then it will be reflected in this value. |
|       purpose       |                                                           A category provided by the borrower for the loan request.                                                          |
|         term        |                                             The number of payments on the loan. Values are in months and can be either 36 or 60.                                             |
|        grade        |                                                                            LC assigned loan grade                                                                            |

### Relación entre `loan_status` otras variables (25 mts)

Consideremos las variables continuas primero

### Ejercicio 1: (5 mts)

Haga 3 diagramas de caja comparando `annual_inc`, `int_rate` y `loan_amnt` contra `loan_status`

**Respuesta.** Al final de la función boxplot, utilice .set_yscale('log')

```python
sns.boxplot(y="annual_inc", x="loan_status", data = df).set_yscale('log');
```

Aquellos que pagan tienen ligeramente más ingresos en promedio

```python
sns.boxplot(y="int_rate", x ='loan_status', data = df);
```

La diferencia es más drástica teniendo en cuenta la tasa de interés.

```python
sns.boxplot(y="loan_amnt", x ='loan_status', data = df)
```

Los usuarios que pagan la deuda parecen contar con montos menores

We learned in previous hypothesis testing cases that we can use contingency tables to examine the relationship between two discrete variables. A chi-square test can be performed based on the contigency table to verify if the observed relationship is statistically significant. Let's examine all discrete variables in our dataset.

### Ejercicio 2: (10 mts)

La siguiente función contruye una tabla de contigencia entre una variable `var` y `loan_status`, incluyendo la proporción de usuarios que pagaron la deuda. Esta tabla sirve para comparar la relación con las variables discretas.

**Respuesta.**

```python
def get_ct(df, var):
    ct_res = pd.crosstab(df[var], df['loan_status'], margins=True)
    ct_res["Fully Paid(%)"] = round( ct_res["Fully Paid"]/ct_res["All"]*100, 2)
    return ct_res.drop(columns="All")
```

```python
get_ct( df, 'verification_status')
```

      loan_status
      Charged Off
      Fully Paid
      Fully Paid(%)

      verification_status

      Not Verified
      2050
      14449
      87.58

      Source Verified
      1413
      8406
      85.61

      Verified
      1977
      10410
      84.04

      All
      5440
      33265
      85.94

```python
get_ct( df, 'emp_length' )
```

      loan_status
      Charged Off
      Fully Paid
      Fully Paid(%)

      emp_length

      < 1 year
      641
      3947
      86.03

      1 year
      460
      2787
      85.83

      2 years
      571
      3823
      87.01

      3 years
      558
      3540
      86.38

      4 years
      466
      2978
      86.47

      5 years
      459
      2826
      86.03

      6 years
      309
      1922
      86.15

      7 years
      263
      1512
      85.18

      8 years
      206
      1279
      86.13

      9 years
      159
      1100
      87.37

      10+ years
      1348
      7551
      84.85

      All
      5440
      33265
      85.94

```python
get_ct( df, 'home_ownership' )
```

      loan_status
      Charged Off
      Fully Paid
      Fully Paid(%)

      home_ownership

      RENT
      2752
      15755
      85.13

      MORTGAGE
      2268
      14987
      86.86

      OWN
      402
      2443
      85.87

      OTHER
      18
      80
      81.63

      All
      5440
      33265
      85.94

```python
get_ct( df, 'purpose' )
```

      loan_status
      Charged Off
      Fully Paid
      Fully Paid(%)

      purpose

      car
      155
      1344
      89.66

      credit_card
      516
      4491
      89.69

      debt_consolidation
      2703
      15572
      85.21

      educational
      52
      265
      83.60

      home_improvement
      333
      2554
      88.47

      house
      59
      310
      84.01

      major_purchase
      211
      1905
      90.03

      medical
      101
      569
      84.93

      moving
      84
      475
      84.97

      other
      600
      3238
      84.37

      renewable_energy
      18
      77
      81.05

      small_business
      466
      1320
      73.91

      vacation
      50
      302
      85.80

      wedding
      92
      843
      90.16

      All
      5440
      33265
      85.94

```python
get_ct( df, 'term' )
```

      loan_status
      Charged Off
      Fully Paid
      Fully Paid(%)

      term

      36 months
      3073
      25211
      89.14

      60 months
      2367
      8054
      77.29

      All
      5440
      33265
      85.94

```python
get_ct( df, 'grade' )
```

      loan_status
      Charged Off
      Fully Paid
      Fully Paid(%)

      grade

      A
      558
      9136
      94.24

      B
      1362
      10335
      88.36

      C
      1296
      6642
      83.67

      D
      1097
      4116
      78.96

      E
      710
      2101
      74.74

      F
      317
      718
      69.37

      G
      100
      217
      68.45

      All
      5440
      33265
      85.94

### Ejercicio 3: (5 mts)

Usar el test de chi cuadrado para ver si las asociaciones son estadísticamente significativas

**Respuesta.** Para cada tabla de contingencia:

```python
chi2, p, dof, ex = chi2_contingency(pd.crosstab(df['verification_status'], df['loan_status']))
print('verification_status: p-value of chisquare test =', p)
```

    verification_status: p-value of chisquare test = 6.884998281535999e-17

```python
chi2, p, dof, ex = chi2_contingency(pd.crosstab(df['emp_length'], df['loan_status']))
print('emp_length: p-value of chisquare test =', p)
```

    emp_length: p-value of chisquare test = 0.06419956286286349

```python
chi2, p, dof, ex = chi2_contingency(pd.crosstab(df['home_ownership'], df['loan_status']))
print('home_ownership: p-value of chisquare test =', p)
```

    home_ownership: p-value of chisquare test = 3.094754275730736e-05

```python
chi2, p, dof, ex = chi2_contingency(pd.crosstab(df['purpose'], df['loan_status']))
print('purpose: p-value of chisquare test =', p)
```

    purpose: p-value of chisquare test = 6.53325963877868e-71

```python
chi2, p, dof, ex = chi2_contingency(pd.crosstab(df['term'], df['loan_status']))
print('term: p-value of chisquare test =', p)
```

    term: p-value of chisquare test = 2.7851291583847273e-194

```python
chi2, p, dof, ex = chi2_contingency(pd.crosstab(df['grade'], df['loan_status']))
print('grade: p-value of chisquare test =', p)
```

    grade: p-value of chisquare test = 0.0

Casi todas son significativas salvo `emp_length`, y `verification_status` tiene un resultado contraintuitivo

### Tablas estratificadas de `verification_status` vs. `loan_status` (15 mts)

```python
def get_ct_stra(stra_var):
    ct_stra = pd.crosstab(index=[stra_var,df.verification_status], columns = df.loan_status, margins = True)
    ct_stra["Fully Paid (%)"] = round(ct_stra["Fully Paid"]/ct_stra["All"]*100,2)
    return ct_stra.drop(columns="All").drop("All", level=0)
```

```python
get_ct_stra(df.home_ownership)
```

      loan_status
      Charged Off
      Fully Paid
      Fully Paid (%)

      home_ownership
      verification_status

      RENT
      Not Verified
      1072
      7133
      86.93

      Source Verified
      815
      4332
      84.17

      Verified
      865
      4290
      83.22

      MORTGAGE
      Not Verified
      798
      6064
      88.37

      Source Verified
      487
      3443
      87.61

      Verified
      983
      5480
      84.79

      OWN
      Not Verified
      170
      1210
      87.68

      Source Verified
      110
      622
      84.97

      Verified
      122
      611
      83.36

      OTHER
      Not Verified
      10
      42
      80.77

      Source Verified
      1
      9
      90.00

      Verified
      7
      29
      80.56

```python
get_ct_stra(df.term)
```

      loan_status
      Charged Off
      Fully Paid
      Fully Paid (%)

      term
      verification_status

      36 months
      Not Verified
      1580
      12574
      88.84

      Source Verified
      688
      6132
      89.91

      Verified
      805
      6505
      88.99

      60 months
      Not Verified
      470
      1875
      79.96

      Source Verified
      725
      2274
      75.83

      Verified
      1172
      3905
      76.92

La diferencia en los pagos verificados y no-verificados podría darse por el efecto del término de los préstamos, pero no por el hecho de tener o no vivienda propia. Usuarios con ingreso no verificado parecen ser más propensos a pagar la deuda.

```python
pd.crosstab(df['term'], df['verification_status'])
```

      verification_status
      Not Verified
      Source Verified
      Verified

      term

      36 months
      14154
      6820
      7310

      60 months
      2345
      2999
      5077

## Regresión logística: entendiendo el modelo lineal a salidas binarias (20 mts)

Del test chi cuadrado, sabemos que la variable verification status está marginalmente asociada la probabilidad de pagar. El modelo de regresión logística  es una modificación del modelo estándar lineal que maneja problemas de clasificación (paga la deuda o no). Los Odds de un evento es la tasa de probabilidad exitosa sobre el fracaso. Los Odds se pagar una deuda se definen así:

$$
\text{Odds}(\text{Full payment}) = \frac{p}{1-p}.
$$

Donde $p$ es la probabilidad de pagar.

La función lineal es la siguiente:

\begin{equation}
\log(\text{Odds}) = \beta_0 + \beta_1*\text{verification_status}. \tag{1}
\end{equation}

Codificamos diferentes niveles de `verification_status` usando enteros: "Not Verified" como 0, "Source Verified" como 1 y "Verified" como 2; si $\beta_1$ es positivo, entonces hay una tendencia creciente en la probabilidad de pagar en tanto que la verificación se vuelve más confiable. Si $\beta_1$ es negativo, sucede lo contrario.

### Interpretación numérica (5 mts)

$p$ - valor de cada coeficiente. Es el resultado del siguiente test de hipótesis:

$$
H_0: \beta = 0~vs.~ H_A: \beta \neq 0.
$$

El test determina si diferencias en la probabildiad de pagar están asociadas a cambios de la variable correspondiente.

### Geometric interpretation of logistic regression (5 mts)

```python
x = np.arange(0,100,1)
y = x/2
plt.plot(x, y, label = "Decision boundary");
plt.xlabel("annual_inc/1e4");
plt.ylabel("loan_amnt/1e3");
plt.text(10, 30, "Non-defaulter", fontsize = 18)
plt.text(60, 20, "Defaulter", fontsize = 18)
plt.legend();
```

La forma del umbral de decisión puede ser también una curva

### Visualizar el umbral de decisión (5 mts)

Considere el siguiente modelo para $p$:

$$
\log(p/(1-p)) = \beta_0 + \beta_1*loan\_amnt+\beta_2*annual\_inc+\beta_3*loan\_amnt^2.
$$

El umbral de decisión con valor $p = 0.5$ es

$$
\beta_0 + \beta_1*loan\_amnt+\beta_2*annual\_inc+\beta_3*loan\_amnt^2 = 0.
$$

```python
%matplotlib widget
import ipywidgets as widgets
from ipywidgets import interact

idx_plot = np.random.choice(df.shape[0], size=500, replace = False)
plt_data = df.iloc[idx_plot]

fig, ax = plt.subplots()
#plt.subplots_adjust(left=0.25, bottom=0.35)
plt.scatter(plt_data[plt_data.loan_status=="Fully Paid"].annual_inc/10000, \
            plt_data[plt_data.loan_status=="Fully Paid"].loan_amnt/1000, \
            c = 'blue', label = 'Fully paid')
plt.scatter(plt_data[plt_data.loan_status=="Charged Off"].annual_inc/10000, \
            plt_data[plt_data.loan_status=="Charged Off"].loan_amnt/1000, \
            c = 'red', label = 'Charged off')
plt.xlim(0,40);
plt.legend();

#plot boundary
beta0 = 1
beta1 = -1
beta2 = 1
beta3 = -1

x_plt = np.arange(0,40,1)
y_plt = (-beta0 - beta1*x_plt - beta3*x_plt**2)/(beta2)
plt.xlabel("annual_inc/1e4")
plt.ylabel("loan_amnt/1e3")
l, = plt.plot(x_plt, y_plt, lw = 2, color="k", label = "Current Decision boundary")
plt.ylim(0,60);
plt.legend();

def update(beta0=1, beta1=-1, beta2=1, beta3=-1):
    y_plt = (-beta0 - beta1*x_plt - beta3*x_plt**2)/(beta2)
    l.set_ydata(y_plt);
    fig.canvas.draw_idle();

interact(update, beta0 = widgets.FloatSlider(value=beta0,min=-10,max=10,step=0.5),\
         beta1 = widgets.FloatSlider(value=beta1,min=-10,max=10,step=0.5),
         beta2 = widgets.FloatSlider(value=beta2,min=0.5,max=10,step=0.5),
         beta3 = widgets.FloatSlider(value=beta3,min=-10,max=0,step=0.5));
```

    Canvas(toolbar=Toolbar(toolitems=[('Home', 'Reset original view', 'home', 'home'), ('Back', 'Back to previous …

    interactive(children=(FloatSlider(value=1.0, description='beta0', max=10.0, min=-10.0, step=0.5), FloatSlider(…

```python
%matplotlib inline
```

## Ajuste del modelo logístico (25 mts)

Utilizando el paquete `statmodels`

```python
# Codificar las variables discretas
df_log1 = pd.DataFrame(columns=['verification_status','loan_status'])
df_log1['verification_status'] = df.verification_status.cat.codes
df_log1['loan_status'] = df.loan_status.cat.codes
df_log1['Intercept'] = 1

logit = sm.Logit(df_log1['loan_status'], df_log1[['Intercept','verification_status']])
logit_res = logit.fit()
print( logit_res.summary() )
```

    Optimization terminated successfully.
             Current function value: 0.405007
             Iterations 6
                               Logit Regression Results
    ==============================================================================
    Dep. Variable:            loan_status   No. Observations:                38705
    Model:                          Logit   Df Residuals:                    38703
    Method:                           MLE   Df Model:                            1
    Date:                Mon, 17 Feb 2020   Pseudo R-squ.:                0.002350
    Time:                        21:06:06   Log-Likelihood:                -15676.
    converged:                       True   LL-Null:                       -15713.
    Covariance Type:            nonrobust   LLR p-value:                 8.419e-18
    =======================================================================================
                              coef    std err          z      P>|z|      [0.025      0.975]
    ---------------------------------------------------------------------------------------
    Intercept               1.9469      0.022     88.290      0.000       1.904       1.990
    verification_status    -0.1460      0.017     -8.600      0.000      -0.179      -0.113
    =======================================================================================

Cuando `verfication_status` se incrementa en una undiad, los Odds se pagar se disminuyen en 14%. Dado que el $p$-valor es menor a 0.05, la
tendencia que observamos en la tabla de contigencia no es producto del azar.

El `Pseudo R-squ.` es una métrica similar al `R-Cuadrado.`, cuya interpretación es más ambigua. De todas formas se buscan valores moderados de este.

### Evaluar la codificación de `verification_status` (5 mts)

El código anterior asume que los log-odds $p/(1-p)$ entre personas con niveles consecutivos de `verification_status` es el mismo. Esto es un supuesto fuerte. Ahora usaremos variables dummy para los diferentes niveles. Not Verified será el nivel de referencia.

```python
vs_dummy = pd.get_dummies(df.verification_status, drop_first=True)
vs_dummy.head()
```

      Source Verified
      Verified

      0
      0
      1

      1
      1
      0

      2
      0
      0

      3
      1
      0

      4
      1
      0

```python
#vs_dummy = pd.get_dummies(df.verification_status, drop_first=True )
df_log1 = pd.concat([df_log1,vs_dummy], axis = 1)

logit_dummy = sm.Logit(df_log1['loan_status'], df_log1[['Intercept','Source Verified','Verified']])
logit_dummy_res = logit_dummy.fit()
print( logit_dummy_res.summary() )
```

    Optimization terminated successfully.
             Current function value: 0.405001
             Iterations 6
                               Logit Regression Results
    ==============================================================================
    Dep. Variable:            loan_status   No. Observations:                38705
    Model:                          Logit   Df Residuals:                    38702
    Method:                           MLE   Df Model:                            2
    Date:                Mon, 17 Feb 2020   Pseudo R-squ.:                0.002366
    Time:                        21:11:16   Log-Likelihood:                -15676.
    converged:                       True   LL-Null:                       -15713.
    Covariance Type:            nonrobust   LLR p-value:                 7.140e-17
    ===================================================================================
                          coef    std err          z      P>|z|      [0.025      0.975]
    -----------------------------------------------------------------------------------
    Intercept           1.9528      0.024     82.741      0.000       1.907       1.999
    Source Verified    -0.1696      0.037     -4.558      0.000      -0.242      -0.097
    Verified           -0.2916      0.034     -8.566      0.000      -0.358      -0.225
    ===================================================================================

Los resultados confirman que la probabilidad de pagar decrece a medida que la información del ingreso se vuelve más confiable.

### Evaluando el modelo (15 mts)

El $R^2$ no puede traducirse para evaluar el modelo.

Podemos utilizar la curva ROC:  **receiver operating characteristic (ROC) curve**.

Tasa de verdaderos positivos: `true positive rate` (TPR) y tasa de verdaderos negativos: `true negative rate` (TNR). In our context, TPR es la proporción de gente que pagó la deuda y fue correctamente clasificada, TNR es la proporción de morosos que fueron identificados como tal.

En ocasiones es difícil hacer que ambas medidas sean altas; en este caso es más importante la tasa de verdaderos negativos, puesto que es preferible prestarle a morosos que no prestarle a buenos candidatos.

El área bajo la curva se calcula como el **AUC**, donde cantidades cercanas a la unidad, indican una predicción buena. La curva ROC para una selección al azar es una línea diagonal.

```python
predict_p = logit_res.predict(df_log1[['Intercept','verification_status']])
predict_dummy_p = logit_dummy_res.predict(df_log1[['Intercept','Source Verified','Verified']])
roc_p = roc_curve(df_log1["loan_status"], predict_p)
roc_dummy_p = roc_curve(df_log1["loan_status"], predict_dummy_p)
auc_p = auc( roc_p[0], roc_p[1] )
auc_dummy_p = auc( roc_dummy_p[0], roc_dummy_p[1] )

plt.figure()
lw = 2
plt.plot(roc_p[0], roc_p[1], color='darkorange',
         lw=lw, label='ROC curve for coding scheme 1 (AUC = %0.2f)' % auc_p)
plt.plot(roc_dummy_p[0], roc_dummy_p[1], color='darkgreen',
         lw=lw, label='ROC curve for coding scheme 2 (AUC = %0.2f)' % auc_dummy_p)
plt.plot([0, 1], [0, 1], color='navy', lw=lw, linestyle='--', label='Random guess')
plt.xlim([0.0, 1.0])
plt.ylim([0.0, 1.05])
plt.xlabel('False Positive Rate')
plt.ylabel('True Positive Rate')
plt.title('Classification of Defaulters')
plt.legend(loc="lower right")
plt.show()
```

    Canvas(toolbar=Toolbar(toolitems=[('Home', 'Reset original view', 'home', 'home'), ('Back', 'Back to previous …

Dado que la curva ROC tiene un AUC cerca al 0.5, se confirma que `verification_status` por sí sola no es buen predictor más allá de elegir al azar. Esto también confirma que no hay diferencia entre las codificaciones anteriormente usadas, así que se mantendrá la primera por simplicidad.

### Exercise 7: (5 mts)

En estadística, ¿son preferibles los modelos simples o complejos?

**Respuesta.**

## Agregando variables adicionales al modelo (20 mts)

La presencia de variables adicionales podría arrojar resultados más contundentes:

```python
# Preprocesar las variables, estandarizar y dummy
df_log2 = pd.concat([(df.loan_amnt - df.loan_amnt.mean())/df.loan_amnt.std(),\
                     (df.int_rate - df.int_rate.mean())/df.int_rate.std(),\
                     (df.annual_inc - df.annual_inc.mean())/df.annual_inc.std(),\
                     pd.get_dummies(df.home_ownership, prefix="home", drop_first=True), \
                     pd.get_dummies(df.purpose, prefix="purpose", drop_first=True), \
                     pd.get_dummies(df.grade, prefix="grade",drop_first=True)], axis=1)
df_log2["verification_status"] = df.verification_status.cat.codes
df_log2["emp_length"] = df.emp_length.cat.codes
df_log2["term"] = df.term.cat.codes
df_log2["Intercept"] = 1
```

```python
logit_full1 = sm.Logit(df.loan_status.cat.codes, df_log2)
logit_full1_res = logit_full1.fit()
print(logit_full1_res.summary())
```

    Optimization terminated successfully.
             Current function value: 0.378075
             Iterations 7
                               Logit Regression Results
    ==============================================================================
    Dep. Variable:                      y   No. Observations:                38705
    Model:                          Logit   Df Residuals:                    38676
    Method:                           MLE   Df Model:                           28
    Date:                Mon, 17 Feb 2020   Pseudo R-squ.:                 0.06869
    Time:                        21:18:21   Log-Likelihood:                -14633.
    converged:                       True   LL-Null:                       -15713.
    Covariance Type:            nonrobust   LLR p-value:                     0.000
    ==============================================================================================
                                     coef    std err          z      P>|z|      [0.025      0.975]
    ----------------------------------------------------------------------------------------------
    loan_amnt                     -0.0075      0.019     -0.389      0.697      -0.045       0.030
    int_rate                      -0.4050      0.054     -7.454      0.000      -0.511      -0.299
    annual_inc                     0.3615      0.031     11.641      0.000       0.301       0.422
    home_MORTGAGE                  0.0382      0.035      1.095      0.273      -0.030       0.107
    home_OWN                       0.0209      0.060      0.348      0.728      -0.097       0.138
    home_OTHER                    -0.4454      0.268     -1.660      0.097      -0.971       0.080
    purpose_credit_card            0.0364      0.100      0.363      0.717      -0.160       0.233
    purpose_debt_consolidation    -0.2190      0.091     -2.405      0.016      -0.398      -0.041
    purpose_educational           -0.5663      0.178     -3.186      0.001      -0.915      -0.218
    purpose_home_improvement      -0.1624      0.107     -1.523      0.128      -0.371       0.047
    purpose_house                 -0.3556      0.171     -2.077      0.038      -0.691      -0.020
    purpose_major_purchase        -0.0024      0.114     -0.021      0.983      -0.227       0.222
    purpose_medical               -0.3945      0.141     -2.794      0.005      -0.671      -0.118
    purpose_moving                -0.4147      0.150     -2.766      0.006      -0.708      -0.121
    purpose_other                 -0.4050      0.099     -4.105      0.000      -0.598      -0.212
    purpose_renewable_energy      -0.7242      0.286     -2.530      0.011      -1.285      -0.163
    purpose_small_business        -0.9414      0.105     -8.961      0.000      -1.147      -0.735
    purpose_vacation              -0.4371      0.179     -2.439      0.015      -0.788      -0.086
    purpose_wedding                0.1348      0.142      0.948      0.343      -0.144       0.414
    grade_B                       -0.2733      0.074     -3.709      0.000      -0.418      -0.129
    grade_C                       -0.3549      0.103     -3.448      0.001      -0.557      -0.153
    grade_D                       -0.3775      0.132     -2.860      0.004      -0.636      -0.119
    grade_E                       -0.2973      0.160     -1.859      0.063      -0.611       0.016
    grade_F                       -0.3238      0.194     -1.669      0.095      -0.704       0.057
    grade_G                       -0.1802      0.238     -0.757      0.449      -0.647       0.286
    verification_status            0.0082      0.020      0.411      0.681      -0.031       0.047
    emp_length                    -0.0147      0.004     -3.301      0.001      -0.023      -0.006
    term                          -0.4520      0.037    -12.081      0.000      -0.525      -0.379
    Intercept                      2.6163      0.123     21.216      0.000       2.375       2.858
    ==============================================================================================

### Ejercicio 8: (5 mts)

¿Es correcta la siguiente afirmación?

No hay evidencia que soporte la relación entre la probabilidad de pagar y la variable `verification_status`

**Respuesta**
Correcta, su $p$-valor es mucho mayor al 0.05 de significancia.

## ¿Es necesario incluir? `verification_status`? (10 mts)

`verification_status` No importa para la predicción.

### Exercise 10: (5 mts)

¿Cuál de las siguientes opciones es correcta sobre la variable `verification_status`?

A. Como el $p$-valor es grande, podemos quitar la variable del modelo de clasificación.

B. Aunque el $p$-valor sea grande, su coeficiente de regresión es distinto de cero, por lo que deberíamos considerarla.

C. Debería de hacerse una evaluación final para comparar las predicciones del modelo con y sin `verification_status` antes de decidir removerla.

D. Ninguna de las anteriores.

**Respuesta.** C. Nunca se aconseja tomar decisiones basadas únicamente en el $p$-valor

Comparemos la precisión en la predicción usando la curva ROC:

```python
predict_withvs = logit_full1_res.predict(df_log2)
logit_full_novs = sm.Logit(df.loan_status.cat.codes, df_log2.loc[:, df_log2.columns != 'verification_status'])
predict_novs = logit_full_novs.fit(disp=0).predict(df_log2.loc[:, df_log2.columns != 'verification_status'])

roc_vs = roc_curve(df.loan_status.cat.codes, predict_withvs)
roc_novs = roc_curve(df.loan_status.cat.codes, predict_novs)
auc_vs = auc( roc_vs[0], roc_vs[1] )
auc_novs = auc( roc_novs[0], roc_novs[1])

plt.figure()
lw = 2
plt.plot(roc_vs[0], roc_vs[1], color='darkorange',
         lw=lw, label='With verification_status (AUC = %0.2f)' % auc_vs)
plt.plot(roc_novs[0], roc_novs[1], color='darkgreen',
         lw=lw, label='Without verification_status (AUC = %0.2f)' % auc_novs)
plt.plot([0, 1], [0, 1], color='navy', lw=lw, linestyle='--', label='Random guess')
plt.xlim([0.0, 1.0])
plt.ylim([0.0, 1.05])
plt.xlabel('False Positive Rate')
plt.ylabel('True Positive Rate')
plt.title('Classification of Defaulters')
plt.legend(loc="lower right")
plt.show()
```

    Canvas(toolbar=Toolbar(toolitems=[('Home', 'Reset original view', 'home', 'home'), ('Back', 'Back to previous …

### Exercise 11: (5 mts)

¿Qué es puede concluir del proceso de validación después de observar los AUC de ambos modelos?

**Respuesta.** Los AUC no cambian, y como se prefiere un modelo tan sencillo como sea posible se decide no incluir la variable.

## Conclusiones (5 mts)

Al principio encontramos un resultado contraintuitivo por la falta de variables explicativas. Luego de hacer el fit del modelo completo encontramos la verdadera tendencia.

En resumen, el proceso de verificación de ingresos es irrelevante y puede ser potencialmente removido para simplificar el sistema de préstamos.

## Aprendizajes (5 mts)

Regresión logística similar a lineal, pero para tareas de clasificación. Curva ROC para comparar modelos.

Monelit Data Science/3. lunes/logit/logistica.ipynb