# **ESS314: LAB 7: Magnetics**

**For this lab, you will need to submit this .ipynb file, as well as separate math from Question 3, and drawings or plots from Questions 6 and 9a.** *As usual, questions for submission are in italics.*

Today we will return to GRAVMAG and look at some magnetic data. The magnetic method is quite complicated, so first we will do a few exercises to improve our understanding of the Earth’s magnetic field.

The intensity of the Earth’s magnetic field at some location on the Earth’s surface can be written as a vector $F_{E}$, defined by its inclination i (the angle the vector makes with a horizontal plane) and its declination d (the horizontal angle from geographic north). **Figure 1** below illustrates these concepts. If you find **Figure 1** a little confusing, then **Figure 2** below illustrates these concepts in a different way using a compass needle. Note that Z is positive down.

```python
import numpy as np
import matplotlib.pyplot as plt
from IPython import display
from base64 import b64decode
```

  **Figure 1.** Diagram illustrating vector FE and its constituent vectors, along with declination and inclination angles d and i, respectively.

**Figure 2.** Representation of vector FE and its constituent vectors, along with declination and inclination angles d and i, respectively, using a compass schematic.

The total magnetic field can be broken down into its horizontal $H_E$ and vertical $Z_E$ components and be represented as

$$F_E = \sqrt{ (H_E)^2 + (Z_E)^2  }.$$
 (1)

$H_E$ can be further resolved into northward and eastward components ($X_E$ and $Y_E$, respectively):

$$H_E = \sqrt{ (X_E)^2 + (Y_E)^2  }.$$
 (2)

Combining equations (1) and (2), one can represent the magnetic field using

$$F_E = \sqrt{ (X_E)^2 + (Y_E)^2 + (Z_E)^2 }.$$
 (3)

### **Question 1:**
Use simple trigonometry to derive the following formulas. We will do the expression for $H_E$ together in lab. Use the previous cell to learn how to write equations in Markdown. Markdown follows the LateX conventions. Enter an equation within the ``$ $`` to enter the mathematical environment, and follow LateX guidelines for the notations (operators, greek letters, powers and indices, fractions, roots, sums and integrals, brackets, ...) (https://en.wikibooks.org/wiki/LaTeX/Mathematics)

>a. *Express $H_E$ and $Z_E$ in terms of the angle of inclination and the total field magnitude.*

>b. *Express $X_E$ and $Y_E$ in terms of the total field magnitude and the angles of inclination and
declination.*

>c. *Assume the magnetic field vector is (4, 1, 1). Calculate the inclination and declination
using the formulas you derived above.*

```python

```

If the magnetic field of the earth could be produced by a simple north-south dipole (**Figure 3**), then inclination would be 0° anywhere along the equator, 90° (down) at the north pole and -90° (up) at the south pole.

**Figure 3.** The magnetic field around a sphere produced by a dipole.

However, the Earth’s magnetic field is imperfect. The magnetic equator (where i = 0°) is
irregular (**Figure 4**). Additionally, the inclination is ±90° at the north and south magnetic poles.
This is why **magnetic north does not equal true north.**

**Figure 4.** Global magnetic inclination values. Source: https://www.ngdc.noaa.gov/geomag/WMM/

Therefore, while the Earth’s magnetic field is not perfect, it is close and is often approximated by a magnetic field produced by a dipole tilted 11.5° to the Earth’s axis of rotation. This reproduces 90% of the Earth’s magnetic field, and we can use this approximation for the vertical and horizontal components of the magnetic field at some point.

### **Question 2:**
> *In your own words, describe the distinctions between the geographic poles, the magnetic poles, and the geomagnetic poles.*

A magnetic field is a potential field. The potential $V$ for a monopole is expressed by

\begin{align}
V = -\int_\infty^r \frac{m}{r^2} dr, \label{eq4}\tag{4}
\end{align}

where $m$ is the pole strength and $r$ is the distance from the pole.

### **Question 3:**

>*Solve the integral to get a simpler equation for potential. Show your work.*

submit your math to canvas either integrated in the notebook text (following LateX/Markddown), or as a separate document / image

Without going through extensive derivations, the magnetic potential at point $P$ due to a dipole is

\begin{align}
V = \frac{M\cos\theta}{r^2}, \label{eq5}\tag{4}
\end{align}

where $M$ is the magnetic moment, $\theta$ is the angle from the dipole to point $P$ (90-latitude for a geocentric dipole), and $r$ is the distance between point $P$ and the dipole. By taking the negative derivatives of the potential in spherical coordinates, one could derive the following equations for the radial and tangential components of the magnetic field:

\begin{align}
H_r &= -Z_E = \frac{2M\cos\theta}{r^3}. \label{eq6}\tag{6}\\
H_\theta &= -X_E = \frac{M\sin\theta}{r^3}.\label{eq7}\tag{7}
\end{align}

Because the Earth is spherical, the radial component $H_r$ is equivalent to the vertical field and the tangential component $H_\theta$ is equivalent to the horizontal southward pointing field. The E-W component of a dipole field ($Y_E$) is zero. See **Figure 5** below for a visualization of this.

**Figure 5.** Some important features of the Earth’s magnetic field.

### **Question 4:**

Assume a radius of 6371 km for the Earth and a magnetic moment of the imaginary dipole responsible for the majority of Earth’s field as 8 x 1025 emu.

>a. *Estimate the horizontal $H_e$ and vertical $Z_e$ fields at the 2006 North magnetic pole in nanoTeslas (latitude 80°; Figure 6). Note that $\theta$ is the colatitude (90 – latitude). The radius, $r$, must be expressed in centimeters. The answer you will get is expressed in oersteds. Multiply by 105 to get the answer in nanoTeslas (nT).*

```python
# calculate here USING CODE!!! SHOW YOUR WORK!
```

>b. *Calculate the total field strength in nanoteslas.*

```python
# calculate here USING CODE!!! SHOW YOUR WORK!
```

>c. *How well does it approximate the actual field strength as shown in Figure 6 below?*

answer here

>d. *What does the value in part c represent?*

answer here

### **Question 5:**
>*What is the ratio of the dipole total field strength at the equator to that at the pole?*

**Figure 6.** Values of total intensity for the geomagnetic field (FE) in 2006 for a portion of the Western Hemisphere.

In a magnetic survey we look for perturbations between the observed magnetic field and a reference field. These perturbations are from smaller magnetic fields induced by geologic bodies.

The magnetic field that we actually measure is the sum of Earth’s magnetic field at the point of study and the anomalous field produced by a geologic body or bodies. However, we are interested in the total field anomaly in the direction of Earth’s main field. This is referred to as $F_{AT}$. $F_{AT}$ depends on the vertical and horizontal components of the anomalous field ($Z_A$ and $H_A$, respectively) and the inclination *i*:

\begin{align}
F_{AT} = Z_A\sin{i} + H_A\cos{i}.\label{eq8}\tag{8}
\end{align}

The next few questions are designed to help you gain some basic intuition on what the magnetic field induced in a simple geological body looks like.

### **Question 6:**
At each location indicated below, determine whether the vertical or horizontal components of the anomalous magnetic field will be more important. Illustrate your answers using vectors.

>a. *Near the poles*

answer here

>b. *Near the equator*

answer here

> submit a drawing to Canvas and/or import here. Give a 1-2 sentence explanation for each.

You just proved that the component (vertical or horizontal) that affects the total induced field the most depends on latitude. Latitude also affects the orientation of Earth’s magnetic field, which induces the magnetic field in the geologic body. This gets confusing, because as you change latitude, not only are you seeing different contributions from the horizontal and vertical field in the total field, but you are also seeing changes in the horizontal and vertical fields themselves. **YIKES!**

## Buried Spherical Body

```python
# below we define a function that calculates the anomaly based on a buried magnetic sphere
# function takes the inputs z, R, k, x, inc
# function outputs the values of Z_a, H_a, and F_at

# z is the depth of the sphere in meters
# R is the radius of the sphere in meters
# k is the magnetic susceptibility in cgs emu (cm^(1/2)/s)
# x is the position on the surface in meters (x=0 is above center of sphere)
# inc is the inclination in degrees

# do not alter this part since this is where we define it
# skip ahead to the next cell to call the function

def mag_sphere(z,R,k,x,inc):
    # changes inclination if 0
    if inc == 0: # in DEGREES - changed to radians in script
        inc = 1e-10
    incr = np.deg2rad(inc) # convert degres to radian
    # calculate magnetic field strength
    theta = np.arctan(np.tan(incr)/2) #inclination to colatitude - this is in radians now
    eR = 6.4e8
    M = 8e25
    Z_e = ((2*M*np.cos(theta))/(eR**3))*(10**5)
    H_e = ((M*np.sin(theta))/(eR**3))*(10**5)
    F_e = np.sqrt(Z_e**2+H_e**2)

    Z_a = ((((4/3)*np.pi*R**3*k*F_e)*np.sin(incr))/(x**2+z**2)**(3/2))*(((3*z**2)/(x**2+z**2))-((3*x*z/np.tan(incr))/(x**2+z**2))-1);
    H_a = ((((4/3)*np.pi*R**3*k*F_e)*np.cos(incr))/(x**2+z**2)**(3/2))*(((3*z**2)/(x**2+z**2))-((3*x*z*np.tan(incr))/(x**2+z**2))-1);
    F_a = Z_a*np.sin(incr)+H_a*np.cos(incr)

    fig = plt.plot(x, Z_a, 'b:', label='Vertical')
    fig = plt.plot(x, H_a, 'r:', label='Horizontal')
    fig = plt.plot(x, F_a, 'g-', label='Total')
    plt.xlabel('Position (x)')
    plt.ylabel('Magnetic Field Anomaly (nT)')
    plt.legend()
    plt.show()
```

Now we will use the function mag_sphere to calculate, and plot the magnetif field anomaly components and total force.

```python
# input parameters
z = 5 # depth from the spherical anomaly in meters
R = 1 # radius of the spherical anomaly in meters
k = 0.005 # magnetic susceptibility in cgs emu (cm^(1/2)/s)
x = np.linspace(-50,50,101) # position on the surface in meters (x=0 is above center of sphere)
inc = 120 #degrees # inclination in degrees

```

```python
# calling the function
mag_sphere(z,R,k,x,inc)
```

### **Question 7:**

Use the mag_sphere function to show the horizontal, vertical, and total field curves for each of the following inclinations. For each plot, write a 1-2 sentence explanation describing the information shown in each plot and what it means for the magnetic anomalies you would see at those places on Earth's surface.

>a. *90° (north pole)*

```python
# answer here
```

>b. *0° (Equator)*

```python
# answer here
```

>c. *45° (Salem, OR)*

```python
# answer here
```

## Buried Rectangular Body

Now, open GRAVMAG.
1. Make sure the units are meters and cgs.

2. It will default to gravity, but we want to change this. Go to **View**, uncheck Gravity and check Induced Magnetics.

3. Add a body.

4. From the **Window** menu, open the Section.

5. Change your body so that it is a small rectangle in the middle.
> a. You can do this either by changing the vertices by typing in the **Model Table** or
by clicking and dragging them in the **Section** window.

6. At the top of the **Model Table** you can enter **Latitude** and the **Profile Azimuth**.

> a. The **Profile Azimuth** is the azimuth (in degrees from North) of your magnetic
line. If your magnetic line trends North/South, this will be 0. If your magnetic
line trends East/West, this will be 90.

> b. For now, change the profile azimuth to 0 for a North/South line.

7. By default, GRAVMAG plots the total field values, but you can change the plot to look at the horizontal or vertical components.

> a. Open the **Magnetic Data** window.

> b. Use the drop-down menu on the data table to change between total, horizontal,
or vertical components.

> c. Play around with the latitude a little bit. You will see that the results are similar
to those of a buried sphere, which we already have a basic understanding of
from the exercise above.

### **Question 8:**
Set the latitude to 90° (North pole).
>a. Move the rectangle deeper into the subsurface. *What does this do to the curve?*

>b. Increase the susceptibility. *What does this do to the total field?*

Despite the added complication of the latitude dependence of magnetic data, you can see that manipulating a simple rectangular body in the subsurface affects magnetic measurements similarly to how it affects gravity measurements. This also means that magnetic surveys have the same problem as gravity surveys: there are multiple subsurface models that can fit the same data set, the inverse problem is not unique. Therefore, the effectiveness of a magnetic study depends on whether there is previous geologic knowledge of the area.

## Real Buried Body: Magnetic Survey
Okay, lets finally get to an example. You are in Arizona (Latitude = 34°N) at an archeological site and are looking for old buried walls. Your colleagues already found one wall, composed of basalt (susceptibility = 0.006 emu). Now you are conducting a magnetic survey to find more walls. You take 101 measurements in a North/South trending line. The results can be found in ArchealogicalData.txt. Load these results into GRAVMAG.

### **Question 9:**

Create a subsurface model that fits the magnetic data.
>a. *Take a screenshot of the shallow subsurface configuration. Submit to Canvas and import here.*

import here

>b. *Does this single survey give you any information about the direction the walls are
extending? If not, how could you figure this out?*

answer here

### **Question 10:**

> You performed both an induced magnetic survey and a gravity survey looking for
features in the shallow subsurface. When you look at your data, the induced magnetism curve is completely flat, but the gravity survey reveals a large negative anomaly. *What could this feature be? Why is there no induced magnetics?*

answer here


```python
import numpy as np
import matplotlib.pyplot as plt
from sympy import *
import warnings
warnings.filterwarnings('ignore')

rho = 5  # 自然利率
alpha = 0.5 # 资本份额
beta = 0.99  # 跨期贴现率
sigma = 1  # 消费的跨期替代弹性的倒数
theta = 3/5  # 名义粘性指数（一个季度内价格不变的企业比例）
phi = 5  # 劳动的跨期替代弹性的倒数（其倒数是Fisher劳动供给弹性）
e = 9  # 产品的替代弹性
lam = (1-theta)/theta*(1-beta*theta)*(1-alpha)/(1-alpha+alpha*e);
psi_ya = (1+phi)/(sigma*(1-alpha)+phi+alpha)
k = (sigma+(phi+alpha)/(1-alpha))*lam  # NKPC 参数 ：pi_t = E pi_{t+1} + k x_t + u_t
Theta = k/e  # 福利损失函数中产出缺口相对于通胀率的比重
gamma = Theta/(k**2+Theta*(1+beta))  # \hat{p}_t = \gamma*\hat{p}_{t-1}+\gamma\beta\hat{p}_{t+1} +\kappa\Lambda/Theta \delta/(1-\beta\delta\rho_u)u_t
delta = (1-np.sqrt(1-4*gamma**2*beta))/(2*gamma*beta)  # 由上式迭代得出的价格的一阶自回归系数
Lambda = lam/e**2  # 平衡路径不同于有效路径的前提下，福利损失函数中产出波动的一阶项系数

T = 12  # 总期数
tz = 5  # 在前tz期，自然利率为负，然后回到正常水平rho
xpi = np.zeros((2,T+1))  # 第0期到第12期
er = 4  # 负实际利率

# 一下两个矩阵刻画了产出缺口（这里指同有效产出的缺口，而非同自然产出的缺口）与通胀的动态
A = Matrix([[1,1/sigma],[k,beta+k/sigma]])
B = Matrix([[1/sigma],[k/sigma]])

# 名义和实际利率
r = np.append(np.ones(tz+1)*-er,np.ones(T-tz)*rho)
it = np.append(np.zeros(tz+1),np.ones(T-tz)*rho)

# Case1 : 自由裁量（相机决策）
# 央行每一期单独优化，而不是考虑全时期总体优化，在冲击结束后，产出缺口和通胀都立即回到0
# 由于每一期的决策只取决于当期的影响，因此称“相机决策”
for i in range(tz+1):
    xpi[:,tz-i:tz+1-i] = A@xpi[:,tz+1-i:tz+2-i]-er*B

# Case2 : 可信承诺（前瞻指引）
# 央行最小化损失函数贴现和，尽管有更多时期产出缺口为负，但是显著减小冲击在初期造成的影响，由于效用函数的凹性，总的福利损失比相机决策少。
tc = 7  # 在7期以后才让利率回升
xpi_c = np.zeros((2,T+1))
M = np.linalg.inv(np.array([[-k,1+beta*(1-delta)],[beta*(1-delta)+k**2/Theta,-k/Theta]]))@np.array([[beta*(1-delta),(1-delta)/sigma],[0,(1-delta)/Theta]])
H = Matrix([[1,1/beta/sigma],[k,1/beta*(1+k/sigma)]])
J = np.array([[0,1],[Theta,k]])

# 第tc+1期是一个转折点，这一期的状态变量待定
x = Symbol('x')
y = Symbol('y')

# 利用动态方程（以及初值）迭代求解tc+1期的状态变量
xpi_c = Matrix(xpi_c)
xpi_c[:,tc+1:tc+2] = np.array([[x],[y]])
for i in range(tc-tz):
    xpi_c[:,tc-i:tc+1-i] = A@xpi_c[:,tc+1-i:tc+2-i]+rho*B
for i in range(tz+1):
    xpi_c[:,tz-i:tz+1-i] = A@xpi_c[:,tz+1-i:tz+2-i]-er*B
summ = Matrix(np.zeros_like(xpi_c[:,tc+1:tc+2]))
for j in range(tc+1):
    summ += H**j@J@Matrix(xpi_c[:,tc-j:tc+1-j])
sol = solve(M@summ+xpi_c[:,tc+1:tc+2],[x,y])
xpi_c[:,tc+1:tc+2] = np.array([[sol[x]],[sol[y]]])

# 再求出各期状态变量
for i in range(tc-tz):
    xpi_c[:,tc-i:tc+1-i] = A@xpi_c[:,tc+1-i:tc+2-i]+rho*B
for i in range(tz+1):
    xpi_c[:,tz-i:tz+1-i] = A@xpi_c[:,tz+1-i:tz+2-i]-er*B

# 求出各期拉格朗日参数 并根据tc+1期的参数计算之后各时期的状态变量
xis = Matrix(np.zeros((2,T+2))) # 从-1开始
for i in range(tc+2):
    xis[:,i+1:i+2] = H@xis[:,i:i+1]-J@xpi_c[:,i:i+1]
for i in range(T-tc-1):
    xpi_c[0,tc+2+i] = k*delta**(i+1)/Theta*xis[0,tc+2]
    xpi_c[1,tc+2+i] = (1-delta)*delta**i*xis[0,tc+2]
xpi_c = np.asarray(xpi_c)

# 计算名义利率
it_c = r.copy()
for i in range(len(r)-1):
    it_c[i] += xpi_c[1,i+1]+sigma*(xpi_c[0,i+1]-xpi_c[0,i])

# 画图
fig,axes = plt.subplots(2,2,figsize=(9,7))
axes[0,0].plot(xpi[0],'-o',label='discretion')
axes[0,0].plot(xpi_c[0],'-o',label='commitment')
axes[0,0].axhline(0,alpha=0.3,ls='--')
axes[0,0].legend()
axes[0,0].set_title('Output')

axes[0,1].plot(xpi[1],'-o',label='discretion')
axes[0,1].plot(xpi_c[1],'-o',label = 'commitment')
axes[0,1].axhline(0,alpha=0.3,ls='--')
axes[0,1].set_title('Inflation')

axes[1,0].plot(it,'-o',label = 'discretion')
axes[1,0].plot(it_c,'-o',label='commitment')
axes[1,0].set_title('Nominal rate')

axes[1,1].plot(r,'-o')
axes[1,1].set_title('Natural rate')

for ax in axes.flatten():
    ax.set_xticks(np.linspace(0,T,T/2+1))
```

```python
print(gamma,delta,lam,Lambda,k,Theta,psi_ya)
```

    0.203533338760889 0.21264462224542016 0.027066666666666673 0.0003341563786008231 0.3248000000000001 0.0360888888888889 1.0

```python
theta**4
```

    0.31640625

```python

```


# "행렬분해 추천시스템"

> "d2l.ai 16장의 행렬분해 추천시스템을 구현하고 ndcg 평가 방법을 이해한다."

- toc: true
- badges: true
- author: 단호진
- categories: [recommender]

행렬 분해 알고리즘은 2006년 Netflix 경연에서 탁월한 성과를 보이며 널리 알리지게 되었다. 이 글에서는 [d2l.ai](https://d2l.ai/chapter_recommender-systems/mf.html)에서 소개하는 행렬 분해 내용과 코드를 기반으로 pytorch 위에 구현해 본다. 이 블로그에 포함된 코드는 모범과는 거리가 멀고 임시변통으로 작성한 경우가 많은 점을 미리 알려둔다. 다음 사항을 염두에 두고 코드를 작성하였다.

- d2l mxnet 코드를 pytorch 방식으로 이식
- pytorch_lightening 사용
- lenskit의 데이터 불러오기 및 평가 기능 시용
- ndcg 이해

## MovieLens 데이터 셋

훈련 및 검증 셋의 통계적인 특성이 유사하게 나타난다. 추천 시스템은 시간 축의 정보도 추천 결과에 반영되어야 하나 본 블로그에서는 시간 정보는 무시하겠다.

```python
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
from lenskit.datasets import ML100K
from lenskit.crossfold import sample_rows
%matplotlib inline

ml100k = ML100K('input/ml-100k')
ratings = ml100k.ratings
ratings['user'] = ratings['user'] - 1
ratings['item'] = ratings['item'] - 1
train = ratings.iloc[:80000]
test = ratings.iloc[80000:]
len(train), len(test)
```

    (80000, 20000)

```python
fig, ax = plt.subplots(1, 2, figsize=(12, 6))
sns.countplot(x='rating', data=train, ax=ax[0])
sns.countplot(x='rating', data=test, ax=ax[1]);
```

```python
fig, ax = plt.subplots(1, 2, figsize=(12, 6))
sns.histplot(x='timestamp', data=train, ax=ax[0])
sns.histplot(x='timestamp', data=test, ax=ax[1]);
```

## pytorch 학습을 위한 Dataset 준비

```python
from torch.utils.data import Dataset, DataLoader

class MLDataset(Dataset):
    def __init__(self, df):
        super().__init__()
        self.df = df.drop(columns=['timestamp'])
        self.df = self.df.astype({'user': int, 'item': int})

    def __len__(self):
        return len(self.df)

    def __getitem__(self, index):
        user = self.df.iloc[index, 0]
        item = self.df.iloc[index, 1]
        rating = self.df.iloc[index, 2]
        return {
            'user': user,
            'item': item,
            'rating': rating,
        }
```

```python
ds_tr = MLDataset(train)
ds_va = MLDataset(test)
```

## 16.3. 행렬 분해(Matrix Factorization)

- 협업 필터링 모델(collaborative filtering model)이다.
- 사용자-아이템 행렬을 두개의 저 차원 행렬로 분해하고 $R=PQ^T$, $P \in \mathbb{R}^{m \times k}$, $Q \in \mathbb{R}^{n \times k}$, 바이어스 추가해 주었다.
- 과적합을 막기 위해 $L_2$ 정규화를 추가하였다.

$R \in \mathbb{R}^{m \times n}$

$\hat R_{ui} = p_u q_i^T + b_u + b_i$

\begin{equation}
\text{argmin}_{P, Q, b} \sum ||R_{ui} - \hat R_{ui}||^2 + \lambda (||P||_F^2 + ||Q||_F^2 + b_u^2 + b_i^2)
\end{equation}

```python
import torch
import torch.nn as nn
import torch.nn.functional as F
import pytorch_lightning as pl
import d2l.torch as d2l
```

### 16.3.2. 모델 구현

```python
class MF(pl.LightningModule):
    def __init__(self, num_factors, num_users, num_items, **kwargs):
        super().__init__(**kwargs)
        self.P = nn.Embedding(num_users, num_factors)
        self.Q = nn.Embedding(num_items, num_factors)
        self.user_bias = nn.Embedding(num_users, 1)
        self.item_bias = nn.Embedding(num_items, 1)

    def forward(self, user_id, item_id):
        # in lightning, forward defines the prediction/inference actions
        P_u = self.P(user_id)
        Q_i = self.Q(item_id)
        b_u = self.user_bias(user_id)
        b_i = self.item_bias(item_id)
        hat_R = (P_u * Q_i).sum(axis=1) + torch.squeeze(b_u) + torch.squeeze(b_i)
        return hat_R

    def training_step(self, batch, batch_index):
        # training_step defines the train loop. It is independent of forward
        users = batch['user']
        items = batch['item']
        ratings = batch['rating']
        hat_R = self.forward(users, items)
        loss = F.mse_loss(hat_R, ratings)
        self.log('train_loss', loss)
        return loss

    def validation_step(self, batch, batch_index):
        users = batch['user']
        items = batch['item']
        ratings = batch['rating']
        hat_R = self.forward(users, items)
        loss = F.mse_loss(hat_R, ratings)
        self.log('val_loss', loss)

    def configure_optimizers(self):
        optimizer = torch.optim.Adam(self.parameters(), lr=0.002, weight_decay=1e-5)
        return optimizer
```

### 16.3.4. 훈련 및 평가

```python
num_users, num_items = ratings['user'].nunique(), ratings['item'].nunique()
print(num_users, num_items)
mf = MF(50, num_users, num_items)
trainer = pl.Trainer(
    gpus=1,
    max_epochs=150,
    callbacks=[pl.callbacks.EarlyStopping('val_loss')],  # Need to study various callbacks
)
```

    GPU available: True, used: True
    TPU available: False, using: 0 TPU cores
    IPU available: False, using: 0 IPUs

    943 1682

```python
trainer.fit(
    mf,
    DataLoader(ds_tr, batch_size=512, shuffle=True, num_workers=12),
    DataLoader(ds_va, batch_size=512, num_workers=12))
```

    LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
    2021-07-31 12:32:02.771252: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudart.so.11.0

      | Name      | Type      | Params
    ----------------------------------------
    0 | P         | Embedding | 47.1 K
    1 | Q         | Embedding | 84.1 K
    2 | user_bias | Embedding | 943
    3 | item_bias | Embedding | 1.7 K
    ----------------------------------------
    133 K     Trainable params
    0         Non-trainable params
    133 K     Total params
    0.535     Total estimated model params size (MB)

    Validation sanity check: 0it [00:00, ?it/s]

    Training: -1it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

    Validating: 0it [00:00, ?it/s]

## NDCG(normalized discounted cumulative gain) - Lenskit 사용

```python
users = test.user.unique()
users = users.astype(int)
recs = []
mf.eval()
for u in users:
    scores = mf(
        torch.ones(num_items, dtype=int) * u,
        torch.arange(num_items)
    )
    scores = scores.detach().numpy()
    scores = pd.DataFrame({
        'user': u,
        'item': np.arange(num_items),
        'score': scores,
    })
    scores = scores.sort_values('score', ascending=False)
    scores = scores.iloc[:100]
    recs.append(scores)
```

```python
recs = pd.concat(recs, ignore_index=True)
recs.tail()
```

      user
      item
      score

      92695
      363
      290
      3.985114

      92696
      363
      989
      3.981152

      92697
      363
      268
      3.979286

      92698
      363
      54
      3.978426

      92699
      363
      78
      3.964217

```python
from lenskit import topn

rla = topn.RecListAnalysis()
rla.add_metric(topn.ndcg)
results = rla.compute(recs, test)
results.head()
```

      nrecs
      ndcg

      user

      862
      100.0
      0.167055

      760
      100.0
      0.028844

      827
      100.0
      0.086463

      888
      100.0
      0.221742

      847
      100.0
      0.144720

```python
sns.histplot(x='ndcg', data=results);
```

```python
print(f'mean ndcg: {results.ndcg.mean():.4f}')
```

    mean ndcg: 0.1070

## NDCG(normalized discounted cumulative gain) - 직접 구현

추천 내용에 알려진 rating을 부여하여 dcg를 계산한다. Ideal dcg는 알려진 rating을 정렬하여 dcg를 계산하고 그 비율을 ndcg로 본다. 점수를 충분히 매긴 검정 세트가 존재해야 더 나은 ndcg를 얻을 수 있는 듯 하다. ndcg는 다른 알고리즘과 비교에 사용할 수 있다.

```python
ndcgs = []

for u in test['user'].unique():
    test_u = test[test['user'] == u].copy()
    recs_u = recs[recs['user'] == u].copy()
    recs_u = recs_u.join(test_u.set_index('item'), on='item', lsuffix='_r', rsuffix='_t')
    recs_u['rating'] = np.nan_to_num(recs_u['rating'])
    recs_u['dcg'] = 1.0 / np.log2(np.arange(2, len(recs_u) + 2))
    dcg = np.dot(recs_u.rating, recs_u.dcg)
    test_u = test_u.sort_values('rating', ascending=False)
    test_u['ideal'] = 1.0 / np.log2(np.arange(2, len(test_u) + 2))
    ideal = np.dot(test_u.rating, test_u.ideal)
    ndcgs.append((u, dcg/ideal))
```

```python
users, ndcg = zip(*ndcgs)
ndcgs = pd.DataFrame({'user': users, 'ndcg': ndcg})
ndcgs.tail()
```

      user
      ndcg

      922
      228
      0.000000

      923
      244
      0.000000

      924
      60
      0.000000

      925
      96
      0.093113

      926
      363
      0.000000

```python
ndcgs.join(results, on='user', lsuffix='_l', rsuffix='_r')
```

      user
      ndcg_l
      nrecs
      ndcg_r

      0
      862
      0.173109
      100.0
      0.167055

      1
      760
      0.032809
      100.0
      0.028844

      2
      827
      0.093277
      100.0
      0.086463

      3
      888
      0.211704
      100.0
      0.221742

      4
      847
      0.150294
      100.0
      0.144720

      ...
      ...
      ...
      ...
      ...

      922
      228
      0.000000
      100.0
      0.000000

      923
      244
      0.000000
      100.0
      0.000000

      924
      60
      0.000000
      100.0
      0.000000

      925
      96
      0.093113
      100.0
      0.076105

      926
      363
      0.000000
      100.0
      0.000000

927 rows × 4 columns

```python
sns.histplot(x='ndcg', data=ndcgs);
```

```python
ndcgs.ndcg.mean(), results.ndcg.mean()
```

    (0.11535802559380859, 0.10698967271109275)

Lenskit과 직접 구현한 ndcg가 비슷한 결과를 보이고 있다.

## 맺으며

추천 시스템에 발가락을 담가 보았다. 알고리즘 평가 방법을 더 깊게 고민해 봐야겠다.


# **Finding Paths in Graphs Using Grover's Algorithm**

Grover's Algorithm is a quantum database search algorithm [1]. Finding one or more solutions to a problem can be solved in $O(\sqrt{N}/t)$ steps, where $t$ is the number of solutions. In this tutorial, you will be shown how to find paths from one vertex to another in an undirected graph $G(V,E)$. If the state is prepared and measured $N$ times, the probability distribution shows all possible paths from one vertex to another. The solutions found are paths where none of the vertices is visited more than once.

##### **References:**

[1]: GROVER, Lov. A fast quantum mechanical algorithm for database search. Título da Revista, Bell Labs, Murray Hill NJ, May, 1996. Available in: arXiv:quant-ph/9605043.

### **The Problem**

Given a graph $G(v,e)$ with $v$ vertices and $e$ edges. The objective of this work is to use Grover's algorithm to search all paths between two vertices of $G(v,e)$ with time complexity $O(\sqrt{v} /t)$. The encoding can be done so that the edges and vertices are represented as qubits in different registers. An untraveled edge remains in state $'0'$ while the traversed edge reverses its state to $'1'$. The vertices, on the other hand, invert their states if the edge connecting them has been traversed. Thus, if a path is traversed in the graph, the vertices in state $'1'$ will be just the starting and ending points of the path. The following image illustrates why the previous statement is valid.

The graph on the far right shows us the solution to the problem. The state of the edges $'0011011'$ shows us the Hamiltonian path, while the states of the vertices $'10100'$ show us what the final configuration of the vertices is for the path to be found. We see that, in a graph with $n$ vertices, the Hamiltonian path passes through $n−1$ edges.

### **Running the algorithm**

First, we need to specify how many vertices and edges the graph has. Two registers must be created so that: In the register where the measurements will be taken, each qubit represents an edge. In the other register, each qubit represents a vertex.

```python
import qiskit
from qiskit import *
from qiskit.visualization import *

# define the variables vertices (v) and edges (e)
v, e = 5, 7

edge = QuantumRegister(e, 'edge')
vertex = QuantumRegister(v, 'vertex')
ancilla = QuantumRegister(1, 'ancilla')
cr = ClassicalRegister(e, 'cr')
qc = QuantumCircuit(edge,vertex,ancilla,cr)
```

```python
def initialize():
    init = QuantumCircuit(edge,vertex,ancilla)
    init.h(range(len(edge)))
    init.x(ancilla[0])
    init.h(ancilla[0])
    init.barrier()
    return init
```

We start by adding all the $2^m$ possible paths that exist in the graph. The way to do this is to apply the operation $H^{\otimes m}$ on the qubits representing the edges

$$|\psi\rangle = \frac{1}{\sqrt{2^m}} \sum |x\rangle \otimes |0\rangle^{\otimes n}|-\rangle$$

```python
initialize().draw('mpl', scale=0.5)
```

We can code the graph as a quantum circuit, using CX operations, so that if the control is set to '1' (traveled path), the target reverses its state (vertices). In this way, we will have the following resulting state

\begin{equation}
|\psi\rangle = \frac{1}{\sqrt{2^m}} \sum |x\rangle \otimes |y\rangle
\end{equation}

where $y_i$ is the vertex configuration for a path $x_i$ traversed. Now, let's create the function that encodes the graph as a quantum circuit that works with the state changes at the vertices according to the edges traversed. The graph we'll use as an example in this tutorial is the one in the following image, where the index of each vertex and each edge corresponds to the qubits of the respective registers

```python
def graph_encoding():

    graph = QuantumCircuit(edge,vertex)

    graph.cx(edge[0],vertex[0])
    graph.cx(edge[0], vertex[1])

    graph.cx(edge[1],vertex[0])
    graph.cx(edge[1], vertex[3])

    graph.cx(edge[2],vertex[1])
    graph.cx(edge[2], vertex[3])

    graph.cx(edge[3],vertex[1])
    graph.cx(edge[3], vertex[2])

    graph.cx(edge[4],vertex[2])
    graph.cx(edge[4], vertex[3])

    graph.cx(edge[5],vertex[3])
    graph.cx(edge[5], vertex[4])

    graph.cx(edge[6],vertex[0])
    graph.cx(edge[6], vertex[4])

    graph.barrier()

    return graph
```

```python
graph_encoding().draw('mpl', scale=0.5, fold=-1)
```

```python
qc = qc.compose(initialize())
qc = qc.compose(graph_encoding())
qc.draw('mpl', scale=0.5, fold=-1)
```

```python
#flip phase of the solutions
def flip():
    fp = QuantumCircuit(edge, vertex, ancilla)
    fp.x([vertex[1],vertex[3],vertex[4]])
    #fp.x([vertex[0],vertex[1],vertex[2],vertex[3],vertex[4]])
    fp.mct([vertex[i] for i in range(len(vertex))], ancilla[0])
    #fp.x([vertex[0],vertex[1],vertex[2],vertex[3],vertex[4]])
    fp.x([vertex[1],vertex[3],vertex[4]])
    fp.barrier()
    return fp
```

The phase inversion operation is intended to invert the phase of the sought state HP. The HP is not known, but we know the final vertex state for a HP, as we saw in the example. In order to reverse the phase of the mapped edges state, we can apply the operation in the vertex register.

\begin{equation}
  |\psi\rangle = \frac{1}{\sqrt{2^b}} \sum |path_x\rangle \otimes (-1)|10100\rangle = \frac{1}{\sqrt{2^ b}} \sum (-1)|path_x\rangle \otimes|10100\rangle
\end{equation}

```python
qc = qc.compose(flip())
qc.draw('mpl',scale=0.5,fold=-1)
```

```python
qc = qc.compose(graph_encoding())
```

```python
#Diffuser operator to perform an inversion about the mean
def diffuser():
    dif = QuantumCircuit(edge)
    dif.h(range(len(edge)))
    dif.x(range(len(edge)))
    dif.h(edge[-1])
    dif.mct([edge[i] for i in range(len(edge)-1)], edge[-1])
    dif.h(edge[-1])
    dif.x(range(len(edge)))
    dif.h(range(len(edge)))
    dif.barrier()
    return dif
```

```python
#Creating Grover iterations
qc = qc.compose(diffuser())

qc.barrier()
qc = qc.compose(graph_encoding())
qc = qc.compose(flip())
qc = qc.compose(graph_encoding())
qc = qc.compose(diffuser())

qc = qc.compose(graph_encoding())
qc = qc.compose(flip())
qc = qc.compose(graph_encoding())
qc = qc.compose(diffuser())
```

```python
qc.draw('mpl', scale=0.5, fold=-1)
```

```python
qc.measure(range(len(edge)), range(len(edge)))
```

    <qiskit.circuit.instructionset.InstructionSet at 0x7f45ab016910>

```python
simulator = Aer.get_backend('qasm_simulator')
counts = execute(qc, backend=simulator, shots=1024).result().get_counts(qc)

```

```python
# removing paths with more than n-1 steps
oneslist = list()
for key, value in counts.items():
    if key.count('1') >= 5: oneslist.append(key)

for i in range(len(oneslist)):  del counts[oneslist[i]]
```

```python
import matplotlib.pyplot as plt
plot_histogram(counts, figsize=(16,8))
```

# Conclusion

We show that it is possible to find paths in graphs using Grover's algorithm. Paths that travel more than $n-1$ (where $n$ is the number of vertices in the graph) edges can be disregarded in the measurements, as they represent paths with repeated vertices. The implemented algorithm demonstrates quadratic speed up compared to a classic brute force algorithm.

```python
import qiskit.tools.jupyter
%qiskit_version_table
%qiskit_copyright
```

Version Information

Qiskit Software	Version

Qiskit	0.27.0

Terra	0.17.4

Aer	0.8.2

Ignis	0.6.0

Aqua	0.9.2

IBM Q Provider	0.14.0

System information

Python	3.7.10 (default, May  3 2021, 02:48:31)
[GCC 7.5.0]

OS	Linux

CPUs	1

Memory (Gb)	12.688987731933594

Thu Jul 08 01:58:38 2021 UTC

This code is a part of Qiskit

© Copyright IBM 2017, 2021.

This code is licensed under the Apache License, Version 2.0. You may
obtain a copy of this license in the LICENSE.txt file in the root directory
 of this source tree or at http://www.apache.org/licenses/LICENSE-2.0.
Any modifications or derivative works of this code must retain this
copyright notice, and modified files need to carry a notice indicating
that they have been altered from the originals.


# Test Shor's algorithm with Cirq

Setup requirements, import and verify required libraries. Source links and more information at the end of this notebook.

```
try:
    import cirq
except ImportError:
    print("installing cirq...")
    !pip install --quiet cirq
    print("installed cirq.")
```

```
import fractions
import math
import random
import matplotlib.pyplot as plt
import numpy as np
import sympy
from typing import Callable, List, Optional, Sequence, Union

import cirq
#import cirq_google

print(cirq.google.Foxtail)
# verify cirq, should print a circuit as shown belown
# (0, 0)───(0, 1)───(0, 2)───(0, 3)───(0, 4)───(0, 5)───(0, 6)───(0, 7)───(0, 8)───(0, 9)───(0, 10)
# │        │        │        │        │        │        │        │        │        │        │
# │        │        │        │        │        │        │        │        │        │        │
# (1, 0)───(1, 1)───(1, 2)───(1, 3)───(1, 4)───(1, 5)───(1, 6)───(1, 7)───(1, 8)───(1, 9)───(1, 10)
```

    (0, 0)───(0, 1)───(0, 2)───(0, 3)───(0, 4)───(0, 5)───(0, 6)───(0, 7)───(0, 8)───(0, 9)───(0, 10)
    │        │        │        │        │        │        │        │        │        │        │
    │        │        │        │        │        │        │        │        │        │        │
    (1, 0)───(1, 1)───(1, 2)───(1, 3)───(1, 4)───(1, 5)───(1, 6)───(1, 7)───(1, 8)───(1, 9)───(1, 10)

    /usr/local/lib/python3.7/dist-packages/ipykernel_launcher.py:12: DeprecationWarning: cirq.google was used but is deprecated.
     it will be removed in cirq v0.14.
     Use cirq_google instead.

      if sys.path[0] == '':

# Understand Order Finding

## Case A: Classical Order Finding

A function for classically computing the order $r$ of an element $x \in \mathbb{Z}_n$ is provided below. This function simply computes the sequence

$$ x^2 \text{ mod } n $$
$$ x^3 \text{ mod } n $$
$$ x^4 \text{ mod } n $$
$$ \vdots $$

until an integer $r$ is found such that $x^r = 1 \text{ mod } n$. Since $|\mathbb{Z}_n| = \phi(n)$, this algorithm for order finding has time complexity $O(\phi(n))$ which is inefficient. (Roughly $O(2^{L / 2})$ where $L$ is the number of bits in $n$.)

```
def classical_order_finder(x: int, n: int) -> Optional[int]:

    # Make sure x is both valid and in Z_n.
    if x < 2 or x >= n or math.gcd(x, n) > 1:
        raise ValueError(f"Invalid x={x} for modulus n={n}.")

    # Determine the order.
    r, y = 1, x
    while y != 1:
        y = (x * y) % n
        r += 1
    return r
```

An example of classically computing $r$ for a given $x \in \mathbb{Z}_n$ and given $n$ is shown in the code block below.

```
n = 15  # The multiplicative group is [1, 2, 4, 7, 8, 11, 13, 14].
x = 8
r = classical_order_finder(x, n)

# Check that the order is indeed correct.
print(f"x^r mod n = {x}^{r} mod {n} = {x**r % n}")
```

    x^r mod n = 8^4 mod 15 = 1

The quantum part of Shor's algorithm is order finding, but done via a quantum circuit, which is discussed below.

Quantum order finding is quantum phase estimation with [unitary $U$](https://en.wikipedia.org/wiki/Unitary_matrix) that computes the modular exponential function $f_x(z)$ for some randomly chosen $x \in \mathbb{Z}_n$. In this tutorial, use arithmetic operations in Cirq which can implement such a unitary $U$. To start we define and use a Modular Expontial operation.

### Define Modular Exponential

The key part of the code block below is the `apply` method which defines the arithmetic operation.

The `apply` method evaluates `(target * base**exponent) % modulus`. The `target` and the `exponent` depend on the values of the respective qubit registers, and the `base` and `modulus` are constant -- namely, the `modulus` is $n$ and the `base` is some $x \in \mathbb{Z}_n$.

```
class ModularExp(cirq.ArithmeticOperation):

    def __init__(
        self,
        target: Sequence[cirq.Qid],
        exponent: Union[int, Sequence[cirq.Qid]],
        base: int,
        modulus: int
    ) -> None:
        if len(target) < modulus.bit_length():
            raise ValueError(f'Register with {len(target)} qubits is too small '
                             f'for modulus {modulus}')
        self.target = target
        self.exponent = exponent
        self.base = base
        self.modulus = modulus

    def registers(self) -> Sequence[Union[int, Sequence[cirq.Qid]]]:
        return self.target, self.exponent, self.base, self.modulus

    def with_registers(
            self,
            *new_registers: Union[int, Sequence['cirq.Qid']],
    ) -> cirq.ArithmeticOperation:
        if len(new_registers) != 4:
            raise ValueError(f'Expected 4 registers (target, exponent, base, '
                             f'modulus), but got {len(new_registers)}')
        target, exponent, base, modulus = new_registers
        if not isinstance(target, Sequence):
            raise ValueError(
                f'Target must be a qubit register, got {type(target)}')
        if not isinstance(base, int):
            raise ValueError(
                f'Base must be a classical constant, got {type(base)}')
        if not isinstance(modulus, int):
            raise ValueError(
                f'Modulus must be a classical constant, got {type(modulus)}')
        return ModularExp(target, exponent, base, modulus)

    def apply(self, *register_values: int) -> int:
        assert len(register_values) == 4
        target, exponent, base, modulus = register_values
        if target >= modulus:
            return target
        return (target * base**exponent) % modulus

    def _circuit_diagram_info_(
            self,
            args: cirq.CircuitDiagramInfoArgs,
    ) -> cirq.CircuitDiagramInfo:
        assert args.known_qubits is not None
        wire_symbols: List[str] = []
        t, e = 0, 0
        for qubit in args.known_qubits:
            if qubit in self.target:
                if t == 0:
                    if isinstance(self.exponent, Sequence):
                        e_str = 'e'
                    else:
                        e_str = str(self.exponent)
                    wire_symbols.append(
                        f'ModularExp(t*{self.base}**{e_str} % {self.modulus})')
                else:
                    wire_symbols.append('t' + str(t))
                t += 1
            if isinstance(self.exponent, Sequence) and qubit in self.exponent:
                wire_symbols.append('e' + str(e))
                e += 1
        return cirq.CircuitDiagramInfo(wire_symbols=tuple(wire_symbols))
```

Use a total number of qubits of $3 (L + 1)$ where $L$ is the number of bits needed to store the integer $n$ to factor. The size of the unitary which implements the modular exponential is thus $4^{3(L + 1)}$. For $n = 15$, the unitary requires storing $2^{30}$ floating point numbers in memory which is out of reach of most current standard laptops.

```
n = 15
L = n.bit_length()

# The target register has L qubits.
target = cirq.LineQubit.range(L)

# The exponent register has 2L + 3 qubits.
exponent = cirq.LineQubit.range(L, 3 * L + 3)

# Display the total number of qubits to factor this n.
print(f"To factor n = {n} which has L = {L} bits, we need 3L + 3 = {3 * L + 3} qubits.")
```

    To factor n = 15 which has L = 4 bits, we need 3L + 3 = 15 qubits.

### Use ModularExp in a circuit

The quantum part of Shor's algorithm is phase estimation with the unitary $U$ corresponding to the modular exponential operation. The following cell defines a function which creates the circuit for Shor's algorithm using the `ModularExp` operation defined above.

Using this function to visualize the circuit for a given $x$ and $n$ as follows.

```
def make_order_finding_circuit(x: int, n: int) -> cirq.Circuit:
    L = n.bit_length()
    target = cirq.LineQubit.range(L)
    exponent = cirq.LineQubit.range(L, 3 * L + 3)
    return cirq.Circuit(
        cirq.X(target[L - 1]),
        cirq.H.on_each(*exponent),
        ModularExp(target, exponent, x, n),
        cirq.qft(*exponent, inverse=True),
        cirq.measure(*exponent, key='exponent'),
    )

n = 15
x = 7
circuit = make_order_finding_circuit(x, n)
print(circuit)
```

    0: ────────ModularExp(t*7**e % 15)────────────────────────────
               │
    1: ────────t1─────────────────────────────────────────────────
               │
    2: ────────t2─────────────────────────────────────────────────
               │
    3: ────X───t3─────────────────────────────────────────────────
               │
    4: ────H───e0────────────────────────qft^-1───M('exponent')───
               │                         │        │
    5: ────H───e1────────────────────────#2───────M───────────────
               │                         │        │
    6: ────H───e2────────────────────────#3───────M───────────────
               │                         │        │
    7: ────H───e3────────────────────────#4───────M───────────────
               │                         │        │
    8: ────H───e4────────────────────────#5───────M───────────────
               │                         │        │
    9: ────H───e5────────────────────────#6───────M───────────────
               │                         │        │
    10: ───H───e6────────────────────────#7───────M───────────────
               │                         │        │
    11: ───H───e7────────────────────────#8───────M───────────────
               │                         │        │
    12: ───H───e8────────────────────────#9───────M───────────────
               │                         │        │
    13: ───H───e9────────────────────────#10──────M───────────────
               │                         │        │
    14: ───H───e10───────────────────────#11──────M───────────────

Put the exponent register into an equal superposition via Hadamard gates. The $X$ gate on the last qubit in the target register is used for phase kickback. The modular exponential operation performs the sequence of controlled unitaries in phase estimation, then apply the inverse quantum Fourier transform (inv-QFT) to the exponent register and measure to read out the result.

To illustrate the measurement results, sample from a smaller circuit. (Note that in practice we would never run Shor's algorithm with $n = 6$ because it is even. This is just an example to illustrate the measurement outcomes.)

Interpret each measured bitstring as an integer, but what do these integers tell us? In the next section use a classically post-process to interpret them.

```
circuit = make_order_finding_circuit(x=5, n=6)
res = cirq.sample(circuit, repetitions=8)

print("Raw measurements:")
print(res)

print("\nInteger in exponent register:")
print(res.data)

def binary_labels(num_qubits):
    return [bin(x)[2:].zfill(num_qubits) for x in range(2 ** num_qubits)]

# Refactor this plot to use this cell's output
# q = cirq.LineQubit.range(3)
# circuit = cirq.Circuit([cirq.H.on_each(*q), cirq.measure(*q)])
# result = cirq.Simulator().run(circuit, repetitions=100)
# _ = cirq.vis.plot_state_histogram(result, plt.subplot(), title = 'Integer in exponent Register', xlabel = 'Integer', ylabel = 'Count', tick_label=binary_labels(3))
```

    Raw measurements:
    exponent=10110100, 00000000, 00000000, 00000000, 00000000, 00000000, 00000000, 00000000, 00000000

    Integer in exponent register:
       exponent
    0       256
    1         0
    2       256
    3       256
    4         0
    5       256
    6         0
    7         0

### Use Classical post-processing

The integer measured is close to $s / r$ where $r$ is the order of $x \in \mathbb{Z}_n$ and $0 \le s < r$ is an integer. Use the continued fractions algorithm to determine $r$ from $s / r$ then return it if the order finding circuit succeeded, else return `None`.

```
def process_measurement(result: cirq.Result, x: int, n: int) -> Optional[int]:
    # Read the output integer of the exponent register.
    exponent_as_integer = result.data["exponent"][0]
    exponent_num_bits = result.measurements["exponent"].shape[1]
    eigenphase = float(exponent_as_integer / 2**exponent_num_bits)

    # Run the continued fractions algorithm to determine f = s / r.
    f = fractions.Fraction.from_float(eigenphase).limit_denominator(n)

    if f.numerator == 0:
        return None

    r = f.denominator
    if x**r % n != 1:
        return None
    return r
```

The next code block shows an example of creating an order finding circuit, executing it, then using the classical postprocessing function to determine the order. Recall that the quantum part of the algorithm succeeds with some probability. If the order is `None`, try re-running the cell a few times.

```
n = 6
x = 5

print(f"Finding the order of x = {x} modulo n = {n}\n")
measurement = cirq.sample(circuit, repetitions=1)
print("Raw measurements:")
print(measurement)

print("\nInteger in exponent register:")
print(measurement.data)

r = process_measurement(measurement, x, n)
print("\nOrder r =", r)
if r is not None:
    print(f"x^r mod n = {x}^{r} mod {n} = {x**r % n}")
```

    Finding the order of x = 5 modulo n = 6

    Raw measurements:
    exponent=1, 0, 0, 0, 0, 0, 0, 0, 0

    Integer in exponent register:
       exponent
    0       256

    Order r = 2
    x^r mod n = 5^2 mod 6 = 1

## Case B: Quantum Order Finding

### Define Quantum Order Finder

We can now define a streamlined function for the quantum version of order finding using the functions we have previously written. The quantum order finder below creates the circuit, executes it, and processes the measurement result.

You should see that the order of $x = 5$ in $\mathbb{Z}_6$ is $r = 2$. Indeed, $5^2 \text{ mod } 6 = 25 \text{ mod } 6 = 1$.

This completes a quantum implementation of an order finder, and the quantum part of Shor's algorithm.

```
def quantum_order_finder(x: int, n: int) -> Optional[int]:
    if x < 2 or n <= x or math.gcd(x, n) > 1:
        raise ValueError(f'Invalid x={x} for modulus n={n}.')

    circuit = make_order_finding_circuit(x, n)

    measurement = cirq.sample(circuit)
    return process_measurement(measurement, x, n)
```

# Use Order Finding

### Use the Quantum Order Finder

Use this quantum order finder to **complete** Shor's algorithm. In the following code block, add a few pre-processing steps which check if $n$ is even or prime or a prime power.

These steps can be done efficiently with a **classical** computer. Then add a final post-processing step which uses the order $r$ to compute a non-trivial factor $p$ of $n$. This is achieved by computing $y = x^{r / 2} \text{ mod } n$ (assuming $r$ is even), then computing $p = \text{gcd}(y - 1, n)$.

The function `find_factor` uses the `quantum_order_finder` by default, which executes Shor's algorithm.

```
def find_factor_of_prime_power(n: int) -> Optional[int]:
    for k in range(2, math.floor(math.log2(n)) + 1):
        c = math.pow(n, 1 / k)
        c1 = math.floor(c)
        if c1**k == n:
            return c1
        c2 = math.ceil(c)
        if c2**k == n:
            return c2
    return None

def find_factor(
    n: int,
    order_finder: Callable[[int, int], Optional[int]] = quantum_order_finder,
    max_attempts: int = 30
) -> Optional[int]:

    if sympy.isprime(n):
        print("n is prime!")
        return None

    if n % 2 == 0:
        return 2

    c = find_factor_of_prime_power(n)
    if c is not None:
        return c

    for _ in range(max_attempts):
        x = random.randint(2, n - 1)
        c = math.gcd(x, n)

        if 1 < c < n:
            return c

        r = order_finder(x, n)

        if r is None:
            continue

        if r % 2 != 0:
            continue

        y = x**(r // 2) % n
        assert 1 < y < n
        c = math.gcd(y - 1, n)
        if 1 < c < n:
            return c

    print(f"Failed to find a non-trivial factor in {max_attempts} attempts.")
    # return None
```

### Use the Classical Order Finder
The `find_factor` operation uses the`quantum_order_finder` method, which executes Shor's algorithm by defaul. Due to the large memory requirements for classically *simulating* this circuit, use the `classical_order_ finder`.

NOTE: Use Shor's algorithm for $n \le 15$.

```
# test with non-prime numbers only
n = 184572
p = find_factor(n, order_finder=classical_order_finder)
q = n // p
p * q == n

print("If p * q == n is True, then this answer is correct.")
print("The result of " + str(p) + " * " + str(q) + " = "+ str(n) + " is " + str(p * q == n))
```

    If p * q == n is True, then this answer is correct.
    The result of 2 * 92286 = 184572 is True

# Learn More and Sources
##### Copyright 2020 The Cirq Developers

### License

```
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
```

This tutorial presents a pedagogical demonstration of Shor's algorithm. It is a modified and expanded version of [this Cirq example](https://github.com/quantumlib/Cirq/blob/master/examples/shor.py).

    View on QuantumAI

    Run in Google Colab

    View source on GitHub

    Download notebook