Mjerenja u bežičnim sustavima (241)

~ Akademska godina 2021/2022 ~
Ante Lojić Kapetanović
Fakultet elektrotehnike, strojastva i brodogradnje
Sveučilište u Splitu
Split, Hrvatska

**Pregled** $-$ Kroz ove laboratorijske vježbe bavit ćemo se analizom radio kanala pri komunikaciji na ultra visokim frekvencijama (*Ultra High Frequencies*, UHF) i super visokim frekvencijama (*Super High Frequencies*, SHF) u zatvorenom prostoru. Sve bitne značajke radiokomunikacije između predajnika signala (*Signal Hound VSG25A USB Vector Signal Generator*, omnidirekcijska antena) i prijemnika signala (*BB60C - 6GHz Real-time Spectrum Analyzer*, omnidirekcijska antena) pratit ćemo na prijemnoj strani. Nakon prikupljanja relevantnih podataka, podatke ćemo *očistiti* i obraditi za daljnju analizu. Odredit ćemo konstantu propagacije radio valova te statističku distribuciju kanala.

```python
import os

import matplotlib.pyplot as plt
import numpy as np
from scipy.stats import rice, kstest

rootdir = os.path.abspath(os.path.join(os.pardir))
DATASET_PATH = os.path.join(rootdir, 'notebooks', 'data')
```

## Demonstracija mjerenja u praksi

https://ieeexplore.ieee.org/document/8783058

### Prikupljanje podataka o snazi signala na prijemnoj strani
Za demonstraciju, prije početka stvarnih mjerenja komercijalnim programskim paketima navedenima na početku dokumenta, dan je primjer mjerenja kanala koristeći Linux ugrađene alate koji ostvaruju komunikaciju s bežičnom mrežnom karticom u računalu i na taj način mjere (procjenjuju) snagu primljenog signala pri čemu je predajnik pristupna točka s kojom je računalo povezano.

Korišteni *software*:
* `iw` - praćenje stanja na kanalu u realnom vremenu
* `nl80211` - `C`-*header* koji služi kao *firmware* za bežičnu mrežnu karticu u računalu
* `Python 3.x.x` - automatizacija prikupljanja podataka

Primjer skripte:
```python
from datetime import datetime
from subprocess import call as pcall

num_of_readings = 20
time_span = 5
with open('starttime.txt', mode='w') as file:
    file.write(f'Measuring start time: {datetime.now()}\n')
for m in range(n_of_readings):
    pcall('iw wlo1 info | grep channel | cut -d " " -f 2  | tee -a channel_tx.txt', shell=True)
    pcall('iw wlo1 info | grep channel | cut -d " " -f 9 | tee -a freq_tx.txt', shell=True)
    pcall('iwconfig wlo1 | grep Signal | cut -d "-" -f 2 | cut -d " " -f 1 | tee -a power_rx.txt', shell=True)
    time.sleep(time_span)
```

### Pregled podataka
Prostor na kojem se odvijalo mjerenje prikazan je na slici:

Prostor je ograđen je s $4$ nosiva zida, dok su ostali unutarnji zidovi od opeke i šuplji te signal ne trpi preveliku atenuaciju na svom putu do prijemnika.
Sva vrata izrađena su od hrastovog drva s blago sjajnom površinom, a prozori s dvostrukim zastakljenjima bili su zatvoreni dok su metalni kapci bili otvoreni tijekom provedbe mjerenja. Visina stropa je $2.3$ m.

Mjerenje je provedeno na pet točaka gore opisanog prostora, gdje je položaj prijamnika označen brojevima od 1 do 5, dok je predajnik označen trokutnim antenskim simbolom. Predajna antena postavljena je na visinu od $1.05$ m iznad poda, dok su prijemne antene postavljene na relativnim visinama od poda u iznosu od $0.80$ m na položaju 1, $0.70$ m na položaju 2, $0.50$ m na položaju 3, $0.90$ m na položaju 4 i $0.00$ m na položaju 5.

### Obrada mjerenih podataka
Kako bi izbjegli potencijalne probleme u kasnijoj analizi, podaci su očišćeni od pogrešno mjerenih vrijednosti. U primjeru su podaci mjereni kroz ukupno 5 dana, na svakoj mjernoj točki mjerenje je trajalo po 24 sata s razmakom pojedinih mjerenja od 6 sekundi koristeći `Python` skriptu prikazanu poviše.

### Komunikacijski kanal
Osim viševalne prirode primljene snage na promatranim mjernim točkama, prijemna strana je podvrgnuta vremenskim promjenama radio kanala uzrokovana šumom i brojnim drugim negativnim faktorima (kretanje osoba, mijenjanje atmosferskih prilika...). Potrebno je, dakle, kreirati statističku karakterizaciju vremenski promjenjivog WiFi kanala na temelju mjerenja kroz relativno duge vremenske intervale unutar prikazanog mjernog prostora.

```python
def convert_to_watts(p_rx):
    return np.power(10, ((p_rx - 30) / 10))

p_rx_list = [-np.loadtxt(os.path.join(DATASET_PATH, 'clean', 'power_rx1.txt')),
             -np.loadtxt(os.path.join(DATASET_PATH, 'clean', 'power_rx2.txt')),
             -np.loadtxt(os.path.join(DATASET_PATH, 'clean', 'power_rx3.txt')),
             -np.loadtxt(os.path.join(DATASET_PATH, 'clean', 'power_rx4.txt')),
             -np.loadtxt(os.path.join(DATASET_PATH, 'clean', 'power_rx5.txt'))]

p_rx_norm_list = []
p_rx_w_list = []
p_rx_w_norm_list = []
for p_rx in p_rx_list:
    p_rx_norm_list.extend(p_rx / np.mean(p_rx))
    p_rx_w = convert_to_watts(p_rx)
    p_rx_w_list.append(p_rx_w)
    p_rx_w_norm_list.extend(p_rx_w / np.mean(p_rx_w))

i_mp = 0
fig, ax = plt.subplots(nrows=3, ncols=2, figsize=(8,6))
for i_row in range(3):
    for i_col in range(2):
        if i_mp < 5:
            ax[i_row][i_col].plot(p_rx_w_list[i_mp] * 1e6, 'r.', alpha=0.1)
            ax[i_row][i_col].set_xlabel('time point')
            ax[i_row][i_col].set_ylabel('$P_{rx}$ [$\\mu$W]')
            ax[i_row][i_col].set_title(f'Measuring point {i_mp + 1}')
        else:
            ax[i_row][i_col].plot(p_rx_norm_list, 'g.', alpha=0.1)
            ax[i_row][i_col].set_xlabel('time point')
            ax[i_row][i_col].set_ylabel('$P_{rx}$')
            ax[i_row][i_col].set_title(f'Normalized power')
        i_mp += 1
plt.tight_layout()
plt.show()
```

```python
def ecdf(p_rx):
    n = len(p_rx)
    x = np.sort(p_rx)
    y = np.arange(1, n+1) / n
    return x, y

mp = 1
x_emp, cdf_emp = ecdf(p_rx_list[mp-1])
x_gaussian, cdf_gaussian = ecdf(np.random.normal(np.mean(p_rx_list[mp-1]), np.std(p_rx_list[mp-1]), x_emp.size))
x_emp_norm, cdf_emp_norm = ecdf(p_rx_list[mp-1]/np.mean(p_rx_list[mp-1]))
x_gaussian_norm, cdf_gaussian_norm = ecdf(np.random.normal(np.mean(p_rx_list[mp-1]/np.mean(p_rx_list[mp-1])), np.std(p_rx_list[mp-1]/np.mean(p_rx_list[mp-1])), x_emp_norm.size))

fig, ax = plt.subplots(nrows=1, ncols=2, sharey=True, figsize=(10, 4))
ax[0].plot(x_emp, cdf_emp, 'r.', label='rx power cdf')
ax[0].plot(x_gaussian, cdf_gaussian, 'k-', label='Gaussian cdf')
ax[0].set_xlabel('$ - P_{rx}$ [dBm]')
ax[0].legend()
ax[0].grid()
ax[1].plot(x_emp_norm, cdf_emp_norm, 'g.', label='normalized rx power cdf')
ax[1].plot(x_gaussian_norm, cdf_gaussian_norm, 'k-', label='normalized Gaussian cdf')
ax[1].set_xlabel('$P_{rx}$ (normalized)')
ax[1].legend()
ax[1].grid()
fig.suptitle(f'Measuring point {mp}')
plt.show()
```

```python
x_total_emp, cdf_total_emp = ecdf(p_rx_norm_list)
x_total_gaussian, cdf_total_gaussian = ecdf(
    np.random.normal(loc=np.mean(p_rx_norm_list),
                     scale=np.std(p_rx_norm_list),
                     size=x_total_emp.size)
)

b, loc, scale = rice.fit(p_rx_norm_list)
rice_rvs = rice.rvs(b, loc, scale, size=x_total_emp.size)
x_total_rician, cdf_total_rician = ecdf(rice_rvs)

fig, ax = plt.subplots(nrows=1, ncols=2, sharey=True, figsize=(10, 4))
ax[0].plot(x_total_emp, cdf_total_emp, 'r.', label='normalized total rx power cdf')
ax[0].plot(x_total_gaussian, cdf_total_gaussian, 'k-', label='normalized Gaussian cdf')
ax[0].set_xlabel('$P_{rx}$ (normalized)')
ax[0].legend()
ax[0].grid()
ax[1].plot(x_total_emp, cdf_total_emp, 'r.', label='normalized total rx power cdf')
ax[1].plot(x_total_rician, cdf_total_rician, 'k-', label='normalized Rician cdf')
ax[1].set_xlabel('$P_{rx}$ (normalized)')
ax[1].legend()
ax[1].grid()
fig.suptitle('rx power for all measuring points normalized to local mean')
plt.show()
```

### Izračun gubitaka kanala

Primljena snaga singala $P_{rx}$ može se zapisati kao:

\begin{equation}
    P_{Rx} = \frac{P_{eirp} G_{rx}}{L_{f.s.}L_{ex}}
    \label{eq.P_mean}
\end{equation}

gdje je $P_{eirp}$ efektivna snaga izotropnog radijatora,  $G_{rx}$ je *gain* prijemnika, $L_{f.s.}$ je gubitak u slobodnom prostoru definiran kroz Friisovu jednadžbu i $L_{ex}$ predstavlja sve dodatne gubitake.

S obzirom na odnos $L_{f.s.} \sim d^2$, očekuje se da će s porastom kvadrata udaljenosti ukupni gubitci slobodnog prostora biti veći a ukupna snaga na prijemu će biti manja. Za realne uvjete govorimo o zakonu opadanja prijemne snage s obzirom na udaljenost $d$ od predajnika: $P_{rx} \sim 1/d^n$, pri čemu je $n$ koeficijent gubitaka realnog prostora koji ovisi o konfiguraciji istog.

```python
distances = [1.4637, 1.5435, 4.7363, 3.9455, 5.4288]
p_rx_localized_mean = []
p_rx_localized_err = []
for p_rx in p_rx_list:
    p_rx_localized_mean.append(np.mean(p_rx))
    p_rx_localized_err.append(np.std(p_rx))
plt.errorbar(distances, p_rx_localized_mean, p_rx_localized_err,
             marker='o', linestyle='None', color='r')
plt.xlabel('tx-to-rx distance [m]')
plt.ylabel('$P_{rx}$ [dBm]')
plt.grid()
plt.show()
```

Gubitci u realnim uvjetima dani su kroz koeficijent $n$ kojeg dobijemo tako što izračunamo nagib pravca koji predstavlja trend opadanja snage s obzirom na udaljenost predajnika i prijemnika za stvarna mjerenja.

```python
popt = np.polyfit(distances, p_rx_localized_mean, deg=1,
                  w=[sigma for sigma in p_rx_localized_err])
fit = np.poly1d(popt)

plt.errorbar(distances, p_rx_localized_mean, p_rx_localized_err,
             linestyle='None', marker='o', color='r',
             label='rx localized mean power')
plt.plot(distances, fit(distances), 'k-', label='linear fit')
plt.annotate(text=(f'$f(x) = n \\cdot x + b$\n\n'
                   f'n={round(popt[0], 2)}\n'
                   f'b={round(popt[1], 2)}'),
             xy=(3.0, fit(3.0)), xytext=(3.5, fit(3.5)+12),
             verticalalignment='top',
             bbox=dict(boxstyle='round', facecolor='wheat', alpha=0.5),
             arrowprops=dict(arrowstyle="->"))
plt.xlabel('tx-rx distance [m]')
plt.ylabel('$P_{rx}$ [dBm]')
plt.legend()
plt.grid()
plt.show()
```

Proračun koristeći komercijalni paket `Wireless InSite` daje veću primljenu snagu u usporedbi s izmjerenim - to ukazuje na važnost dodatnih prepreka signalu koji nisu uključeni u simulaciju, ali obično se nalaze u stambenim apartmanima (vodljive plohe, elektronički uređaji, dinamika ljudi u prostoru itd.)

Koeficijent propagacije za slobodni prostor dan je kroz izraz:
$$ n_{FS} = \left(\frac{4πdf}{c}\right)^{2} $$

gdje je $d$ udaljenost između prijemnika i predajnika, $f$ je frekvencija, a $c$ je brzina elektromagnetskog vala.
Budući da su gubici definirani kroz dBm, prethodni izraz je potrebno prilagoditi:

$$
\begin{align}
n_{FS}(\text{dB})
  &= 10\log_{10}\left(\left(\frac{4\pi d f}{c}\right)^2\right) \\
  &= 20\log_{10}\left(\frac{4\pi d f}{c}\right)  \\
  &= 20\log_{10}(d) + 20\log_{10}(f) + 20\log_{10}\left(\frac{4\pi}{c}\right) \\
  &= 20\log_{10}(d) + 20\log_{10}(f) - 147.55
\end{align}
$$

```python
wireless_insite_p_rx_localized = np.array([-49.0, -42.0, -52.5, -55.0, -59.0])
log_distances = 10 * np.log10(distances)
popt_wi = np.polyfit(log_distances, wireless_insite_p_rx_localized, deg=1)
fit_wi = np.poly1d(popt_wi)

plt.errorbar(log_distances, p_rx_localized_mean, p_rx_localized_err, linestyle='None', marker='o', color='r', label='measured rx localized mean power')
plt.plot(log_distances, fit(log_distances), 'k-', label='linear fit on measured power')
plt.plot(log_distances, wireless_insite_p_rx_localized, 'go', label = 'Wireless Insite simulated rx power')
plt.plot(log_distances, fit_wi(log_distances), 'k--', label='liner fit on Wireless Insite simulated power')
plt.xlabel('tx-rx distance [m]')
plt.ylabel('$P_{rx}$ [dBm]')
plt.legend()
plt.grid()
plt.show()

print(f'[emprical] n = {np.abs(popt[0])}')
print(f'[simulated] n = {np.abs(popt_wi[0])}')
```

### Izračun gubitaka snage

Ukupni gubici računaju se preko izraza:

$$ a_{f.s.} [\mbox{dB}] = 32.5 + 20 \cdot \log{f [\mbox{MHz}]} + 20 \cdot \log{d [\mbox{km}]} - G_{tx} [\mbox{dBi}] - G_{rx} [\mbox{dBi}]$$

gdje je $f$ frekvencija kanala, $d$ udaljenost između prijemnika i predajnika te $G_{Tx}$ i $G_{Rx}$ dobitci predajnika i prijemnika.

Stvarne gubitke tijekom mjerenja je jednostavno izračunati preko izraza:

$$ a = P_{tx} - P_{rx} $$

gdje je $P_{tx}$ snaga signala na izlazu iz predajnika, a $P_{rx}$ je primljena snaga na prijemniku.

Ako se za istu frekvenciju i udaljenost između prijemnika i predajnika provede veći broj mjerenja, potrebno je takva mjerenja usrednjiti prije uvrštavanja u prethodno dani izraz. Osim toga, potrebno je definirati i stupanj pouzdanosti mjerenja eksplicitno navodeći moguću pogrešku.

```python
f = 2450
p_tx = 10
G_tx = G_rx = 0.725
n = np.abs(popt[0])
d = np.asarray(distances, dtype=np.float32)

a_fs = (32.5
        + (n * 10 * np.log10(f))
        + (n * 10 * np.log10(d / 1000))
        - (G_tx + G_rx))
a_emp = p_tx - np.asarray(p_rx_localized_mean)

print(f'Free space loss approximation: {a_fs}')
print(f'Experimental loss: {a_emp}')
```

    Free space loss approximation: [51.336857 52.18015  69.99121  67.08921  72.159   ]
    Experimental loss: [45.74055171 69.02406596 70.22415982 76.17063669 74.08733533]


```python
%run ../../common/import_all.py

from common.setup_notebook import set_css_style, setup_matplotlib, config_ipython
config_ipython()
setup_matplotlib()
set_css_style()
```

# AdaBoost

## The gist

AdaBoost, shortening for *adaptive boosting* is a boosting ensemble method used both in classification and in regression problems. The authors won the Goedel prize for the work in 2003, whose original paper is in [[1]](#paper) but a nice reading over the general idea, by the same authors is in [[2]](#boosting-gentle).

The idea is to fit a sequence of weak learners (a weak learner is one that is just slightly better than a random guesser) on repeatedly modified versions of the training set, then combine their predictions through a weighted majority voting system.

In the first iteration, you give the same weight to all $n$ training samples, $w_i = \frac{1}{n}$; in successive iterations the weights are modified in such a way that the training samples which were incorrectly predicted in the previous iteration will see their weights increased while those which were correctly predicted will see their weights decreased. This way, the weak learner is forced to focus on the points more difficult to predict: this is the adaptive part of the idea. The resulting combination of these weak learners will be a strong learner.

As per current literature, AdaBoost with decision trees as the learners is considered one of the best classifiers.

## The algorithm

This part follows [the Wikipedia page](https://en.wikipedia.org/wiki/AdaBoost). Let's see we have a binary classification problem, class variables being $y_i \in \{1, -1\}$ and sample points $(x_1, y_1), \ldots, (x_n, y_n)$, where $x_i \in X$ (feature matrix).

* Call a weak learner $h$
* At iteration $t$, you'll have built a combination of weak learners into a strong learner

$$
H_t(x_i) = \sum_{i=1}^t \alpha_i h_i(x_i) \ ,
$$

so that

$$
H_t(x_i) = H_{t-1}(x_i) + \alpha_t h_t(x_i)
$$

This way we have built a linear combination of weak learners over several iterations. The weights $\alpha_i$ of each learner remain to be attributed in the most effective way. If we consider the error $E$ on $H_t$ as the sum of the exponential losses on each data point,

$$
E = \sum_{i=1}^n e^{-y_i H_t(x_i)}
$$

(note that the argument of the exponential will be a 1 if the point is well classified and a -1 if not) which by posing $w_i^1 = 1$ and $w_i^t = e^{-y_i H_{t-1}(x_i)}$ ($w_i$ represents a weight to the error) we can rewrite as

$$
E = \sum_{i=1}^n w_i^t e^{-y_i \alpha_t h_t(x_i)} \ .
$$

We can now split the sum between the points which are well classified and those misclassified:

$$
\begin{align}
    E &= \sum_{i: y_i = h_t(x_i)} w_i^t e^{-\alpha_t} + \sum_{i: y_i \neq h_t(x_i)} w_i^t e^{\alpha^t} \\
      &= \sum_{i=1}^n w_i^t e^{-\alpha_t} + \sum_{i: y_i \neq h_t(x_i)} w_i^t [e^\alpha_t - e^{-\alpha_t}]
\end{align}
$$

In the last expression above, the only part depending on the weak classifiers is the second sum, so the weak classifier that minimises $E$ is the one minimising this sum, which means the one that minimises $\sum_{i: y_i \neq h_t(x_i)} w_i^t$, so the one with the lowest weighted error.

If we derive $E$ with respect to $\alpha_t$, we obtain

$$
\alpha_t = \frac{1}{2} \log{\frac{\sum_{i: y_i = h_t(x_i)} w_i^t}{\sum_{i: y_i \neq h_t(x_i)} w_i^t}}
$$

Now, the weighted error rate of the weak classifiers is

$$
\epsilon_t = \frac{\sum_{i: y_i \neq h_t(x_i)} w_i^t}{\sum_{i=1}^n w_i^t} \ ,
$$

so it follows that we can write the $\alpha_t$ which minimises $E$ as

$$
\alpha_t = \frac{1}{2} \log{\frac{1-\epsilon_t}{\epsilon_t}}
$$

which is $\frac{1}{2}$ times the logit negative function.

To summarise then, the AdaBoost algorithm consists of

1. Choose the weak classifier which minimised the error $E$
2. Use it to compute the classifiers weighted error $\epsilon_t$
3. Use this to compute the weights $\alpha_t$
4. use this to compute the boosted (strong) classifier $H_t$

Note that there exist several variants of the original AdaBoost.

## References

1.  Y Freund, R E Schapire, [**A decision-theoretic generalization of on-line learning and an application to boosting**](http://cns.bu.edu/~gsc/CN710/FreundSc95.pdf), *J of computer and system sciences* 55.1, 1997
2.  Y Freund, R E Schapire, [**A short introduction to boosting**](https://cseweb.ucsd.edu/~yfreund/papers/IntroToBoosting.pdf)

```python

```


# CIS 519 Homework 2: Linear Classifiers (Answers)

- Handed Out: October 5, 2020
- Due: October 19, 2020 at 11:59pm.

Although the solutions are my own, I consulted with the following people
while working on this homework:
- TODO (if applicable): ...

## Preface

- Feel free to talk to other members of the class in doing the homework. I am more concerned that you learn how to solve the problem than that you demonstrate that you solved it entirely on your own. You should, however, **write down your solution yourself**. Please include here the list of people you consulted with in the course of working on the homework:

- While we encourage discussion within and outside the class, cheating and copying code is strictly not allowed. Copied code will result in the entire assignment being discarded at the very least.

- Please use Piazza if you have questions about the homework. Also, please come to the TAs recitations and to the office hours.

- The homework is due at 11:59 PM on the due date. We will be using Gradescope for collecting the homework assignments. You should have been automatically added to Gradescope. If not, please ask a TA for assistance. Post on Piazza and contact the TAs if you are having technical difficulties in submitting the assignment.

- Here are some resources you will need for this assignment (https://www.seas.upenn.edu/~cis519/fall2020/assets/HW/HW2/hw2-materials.zip)

# Overview

### About Jupyter Notebooks

In this homework assignment, we will use a Jupyter notebook to implement, analyze, and discuss ML classifiers.
Knowing and being comfortable with Jupyter notebooks is a must in every data scientist, ML engineer, researcher, etc. They are widely used in industry and are a standard form of communication in ML by intertwining text and code to "tell a story". There are many resources that can introduce you to Jupyter notebooks (they are pretty easy to understand!), and if you still need help any of the TAs are more than willing to help.

We will be using a local instance of Jupyter instead of Colab. You are of course free to use Colab, but you will need to understand how to hook your Colab instance with Google Drive to upload the datasets and to save images.

### About the Homework

You will experiment with several different linear classifiers and analyze
their performances in both real and synthetic datasets. The goal is to understand the differences and
similarities between the algorithms and the impact that the dataset characteristics have on the
algorithms' learning behaviors and performances.

In total, there are seven different learning algorithms which you will implement.
Six are variants of the Perceptron algorithm and the seventh is a support vector machine (SVM).
The details of these models is described in Section 1.

In order to evaluate the performances of these models, you will use several different datasets.
The first two datasets are synthetic datasets that have features and labels that were programatically
generated. They were generated using the same script but use different input parameters that produced
sparse and dense variants. The second two datasets are for the task of named-entity recognition (NER),
identifying the names of people, locations, and organizations within text.
One comes from news text and the other from a corpus of emails.
For these two datasets, you need to implement the feature extraction yourself.
All of the datasets and feature extraction information are described in Section 2.

Finally, you will run two sets of experiments, one on the synthetic data and one on the NER data.
The first set will analyze how the amount of training data impacts model performance.
The second will look at the consequences of having training and testing data that come from different domains.
The details of the experiments are described in Section 3.

### Distribution of Points

The homework has 4 sections for a total of 100 points + 10 extra credit points:
- Section 0: Warmup (5 points)
- Section 1: Linear Classifiers (30 points)
- Section 2: Datasets (0 points, just text)
- Section 3: Experiments (65 points)
    - Synthetic Experiment:
        - Parameter Tuning (10 points)
        - Learning Curves(10 points)
        - Final Test Accuracies (5 points)
        - Discussion Questions (5 points)
        - Noise Experiment (10 points **extra credit**)
    - NER Experiment:
        - Feature Extraction (25 points)
        - Final Test Accuracies (5 points)
        - $F_1$ Discussion Questions (5 points)

# Section 0: Warmup

###### Only For Colab

If you want to complete this homework in Colab, you are more than welcome to.
You will need a little bit more maneuvering since you will need to upload
the files of hw2 to your Google Drive and run the following two cells:

```python
# Uncomment if you want to use Colab for this homework.
# from google.colab import drive
# drive.mount('/content/drive', force_remount=True)
```

```python
# Uncomment if you want to use Colab for this homework.
# cd /content/drive/My Drive/Colab Notebooks/YOUR_PATH_TO_HW_FOLDER
```

###### Python Version

Python 3.6 or above is required for this homework. Make sure you have it installed.

```python
# Let's check.
import sys
if sys.version_info[:2] < (3, 6):
    raise Exception("You have Python version " + str(sys.version_info))
```

## Imports and Helper Functions (5 points total)

Let's import useful modules we will need throughout the homework
as well as implement helper functions for our experiment. **Read and remember** what each function is doing, as you will probably need some of them down the line.

```python
# Install necessary libraries for this homework.
%pip install sklearn
%pip install matplotlib
%pip install numpy
```

    Requirement already satisfied: sklearn in /usr/local/lib/python3.8/site-packages (0.0)
    Requirement already satisfied: scikit-learn in /usr/local/lib/python3.8/site-packages (from sklearn) (0.23.2)
    Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.8/site-packages (from scikit-learn->sklearn) (0.16.0)
    Requirement already satisfied: numpy>=1.13.3 in /usr/local/lib/python3.8/site-packages (from scikit-learn->sklearn) (1.19.0)
    Requirement already satisfied: scipy>=0.19.1 in /usr/local/lib/python3.8/site-packages (from scikit-learn->sklearn) (1.5.2)
    Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.8/site-packages (from scikit-learn->sklearn) (2.1.0)
    Note: you may need to restart the kernel to use updated packages.
    Requirement already satisfied: matplotlib in /usr/local/lib/python3.8/site-packages (3.3.2)
    Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.8/site-packages (from matplotlib) (1.2.0)
    Requirement already satisfied: certifi>=2020.06.20 in /usr/local/lib/python3.8/site-packages (from matplotlib) (2020.6.20)
    Requirement already satisfied: numpy>=1.15 in /usr/local/lib/python3.8/site-packages (from matplotlib) (1.19.0)
    Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.8/site-packages (from matplotlib) (2.8.1)
    Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.3 in /Users/nacho/Library/Python/3.8/lib/python/site-packages (from matplotlib) (2.4.7)
    Requirement already satisfied: pillow>=6.2.0 in /usr/local/lib/python3.8/site-packages (from matplotlib) (7.2.0)
    Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.8/site-packages (from matplotlib) (0.10.0)
    Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.8/site-packages (from python-dateutil>=2.1->matplotlib) (1.14.0)
    Note: you may need to restart the kernel to use updated packages.
    Requirement already satisfied: numpy in /usr/local/lib/python3.8/site-packages (1.19.0)
    Note: you may need to restart the kernel to use updated packages.

```python
import json
import os

import numpy as np
import matplotlib.pylab as plt
from sklearn.feature_extraction import DictVectorizer
from sklearn.metrics import accuracy_score

DATASETS_PATH = "datasets/"
NER_PATH = os.path.join(DATASETS_PATH, 'ner')
SYNTHETIC_PATH = os.path.join(DATASETS_PATH, 'synthetic')
```

```python
"""
Helper function that loads a synthetic dataset from the dataset root (e.g. "synthetic/sparse").
You should not need to edit this method.
"""
def load_synthetic_data(dataset_type):

    def load_jsonl(file_path):
        data = []
        with open(file_path, 'r') as f:
            for line in f:
                data.append(json.loads(line))
        return data

    def load_txt(file_path):
        data = []
        with open(file_path, 'r') as f:
            for line in f:
                data.append(int(line.strip()))
        return data

    def convert_to_sparse(X):
        sparse = []
        for x in X:
            data = {}
            for i, value in enumerate(x):
                if value != 0:
                    data[str(i)] = value
            sparse.append(data)
        return sparse

    path = os.path.join(SYNTHETIC_PATH, dataset_type)

    X_train = load_jsonl(os.path.join(path, 'train.X'))
    X_dev = load_jsonl(os.path.join(path, 'dev.X'))
    X_test = load_jsonl(os.path.join(path, 'test.X'))

    num_features = len(X_train[0])
    features = [str(i) for i in range(num_features)]

    X_train = convert_to_sparse(X_train)
    X_dev = convert_to_sparse(X_dev)
    X_test = convert_to_sparse(X_test)

    y_train = load_txt(os.path.join(path, 'train.y'))
    y_dev = load_txt(os.path.join(path, 'dev.y'))
    y_test = load_txt(os.path.join(path, 'test.y'))

    return X_train, y_train, X_dev, y_dev, X_test, y_test, features
```

```python
"""
Helper function that loads the NER data from a path (e.g. "ner/conll/train").
You should not need to edit this method.
"""
def load_ner_data(dataset=None, dataset_type=None):
    # List of tuples for each sentence
    data = []
    path = os.path.join(os.path.join(NER_PATH, dataset), dataset_type)
    for filename in os.listdir(path):
        with open(os.path.join(path, filename), 'r') as file:
            sentence = []
            for line in file:
                if line == '\n':
                    data.append(sentence)
                    sentence = []
                else:
                    sentence.append(tuple(line.split()))
    return data
```

```python
"""
A helper function that plots the relationship between number of examples
and accuracies for all the models. You should not need to edit this method.
"""
def plot_learning_curves(
    perceptron_accs,
    winnow_accs,
    adagrad_accs,
    avg_perceptron_accs,
    avg_winnow_accs,
    avg_adagrad_accs,
    svm_accs
):
    """
    This function will plot the learning curve for the 7 different models.
    Pass the accuracies as lists of length 11 where each item corresponds
    to a point on the learning curve.
    """

    accuracies = [
        ('perceptron', perceptron_accs),
        ('winnow', winnow_accs),
        ('adagrad', adagrad_accs),
        ('avg-perceptron', avg_perceptron_accs),
        ('avg-winnow', avg_winnow_accs),
        ('avg-adagrad', avg_adagrad_accs),
        ('svm', svm_accs)
    ]

    x = [500, 1000, 1500, 2000, 2500, 3000, 3500, 4000, 4500, 5000, 10000]
    plt.figure()
    f, (ax, ax2) = plt.subplots(1, 2, sharey=True, facecolor='w')

    for label, acc_list in accuracies:
        assert len(acc_list) == 11
        ax.plot(x, acc_list, label=label)
        ax2.plot(x, acc_list, label=label)

    ax.set_xlim(0, 5500)
    ax2.set_xlim(9500, 10000)
    ax2.set_xticks([10000])
    # hide the spines between ax and ax2
    ax.spines['right'].set_visible(False)
    ax2.spines['left'].set_visible(False)
    ax.yaxis.tick_left()
    ax.tick_params(labelright='off')
    ax2.yaxis.tick_right()
    ax2.legend()
    plt.show()

```

### F1 Score (5 points)

For some part of the homework, you will use the F1 score instead of accuracy to evaluate how well a model does. The F1 score is computed as the harmonic mean of the precision and recall of the classifier. Precision measures the number of correctly identified positive results by the total number of positive results. Recall, on the other hand, measures the number of correctly identified positive results divided by the number of all samples that should have been identified as positive. More formally, we have that

$$
\begin{align}
\text{Precision} &= \frac{TP}{TP + FP} \\
\text{Recall} &= \frac{TP}{TP + FN}
\end{align}
$$

where $TP$ is the number of true positives, $FP$ false positives and $FN$ false negatives. Combining these two we define F1 as

$$
\text{F1} = 2 \cdot \frac{\text{Precision} \cdot \text{Recall}}{\text{Precision} + \text{Recall}}
$$

You now need to implement the calculation of F1 yourself using the provided function header. It will be unit tested on Gradescope.

```python
def calculate_f1(y_gold, y_model):
    """
    Computes the F1 of the model predictions using the
    gold labels. Each of y_gold and y_model are lists with
    labels 1 or -1. The function should return the F1
    score as a number between 0 and 1.
    """
    tp, fp, fn = 0, 0, 0
    for y_true, y_pred in zip(y_gold, y_model):
        if y_true == 1:
            if y_pred == 1:
                tp += 1
            else:
                fn += 1
        else:
            if y_pred == 1:
                fp += 1

    precision = tp / (tp + fp)
    recall = tp / (tp + fn)
    return 2 * (precision * recall) / (precision + recall)
```

Looking at the formula for the F1 score, what is the highest and lowest possible value?

```python
def highest_and_lowest_f1_score():
    """
    Return the highest and lowest possible F1 score
    (ie one line solution returning the theoretical max and min)
    """
    return 1, 0
```

# Section 1. Linear Classifiers (30 points total)

This section details the seven different algorithms that you will use in the experiments. For each of the algorithms, we describe the initialization you should use to start training and the different parameter settings that you should use for the experiment on the synthetic datasets. Each of the update functions for the Perceptron, Winnow, and Perceptron with AdaGrad will be unittested on Gradescope, so please do not edit the function definitions.

### 1.1 Base Classifier

```python
class Classifier(object):
    """
    The Classifier class is the base class for all of the Perceptron-based
    algorithms. Your class should override the "process_example" and
    "predict_single" functions. Further, the averaged models should
    override the "finalize" method, where the final parameter values
    should be calculated. You should not need to edit this class any further.
    """

    ITERATIONS = 10

    def train(self, X, y):
        for iteration in range(self.ITERATIONS):
            for x_i, y_i in zip(X, y):
                self.process_example(x_i, y_i)
        self.finalize()

    def process_example(self, x, y):
        """
        Makes a prediction using the current parameter values for
        the features x and potentially updates the parameters based
        on the gradient. Note "x" is a dictionary which maps from the feature
        name to the feature value and y is either 1 or -1.
        """
        raise NotImplementedError

    def finalize(self):
        """Calculates the final parameter values for the averaged models."""
        pass

    def predict(self, X):
        """
        Predicts labels for all of the input examples. You should not need
        to override this method.
        """
        return [self.predict_single(x) for x in X]

    def predict_single(self, x):
        """
        Predicts a label, 1 or -1, for the input example. "x" is a dictionary
        which maps from the feature name to the feature value.
        """
        raise NotImplementedError
```

### 1.2 Basic Perceptron (2 points)

We do this classifier for you, so enjoy the two free points and pay attention to the techniques and code written.

#### 1.2.1 Description

This is the basic version of the Perceptron Algorithm.
    In this version, an update will be performed on the example $(\textbf{x}, y)$ if $y(\mathbf{w}^\intercal \mathbf{x} + \theta) \leq 0$. The Perceptron needs to learn both the bias term $\theta$ and the weight vector $\mathbf{w}$ parameters.
When the Perceptron makes a mistake on the example $(\textbf{x}, y)$, both $\mathbf{w}$ and $\theta$ need to be updated using the following update equations:
$$
\begin{align*}
    \mathbf{w}^\textrm{new} &\gets \mathbf{w} + \eta \cdot y \cdot \mathbf{x} \\
    \theta^\textrm{new} &\gets \theta + \eta \cdot y
\end{align*}
$$
where $\eta$ is the learning rate.

#### 1.2.2 Hyperparameters

We set $\eta$ to 1, so there are no hyperparameters to tune.

Note: If we assume that the order of the examples presented to the algorithm is fixed, we initialize $\mathbf{w} = \mathbf{0}$ and $\theta = 0$, and train both together, then the learning rate $\eta$ does not have any effect.
        In fact you can show that, if $\mathbf{w}_1$ and $\theta_1$ are the outputs of the Perceptron algorithm with learning rate $\eta_1$, then $\mathbf{w}_1/\eta_1$ and $\theta_1/\eta_1$ will be the result of the Perceptron with learning rate 1 (note that these two hyperplanes give identical predictions).

#### 1.2.3 Initialization

$\mathbf{w} = [0, 0, \dots, 0]$ and $\theta = 0$.

```python
class Perceptron(Classifier):
    """
    The Perceptron model. Note how we are subclassing `Classifier`.
    """

    def __init__(self, features):
        """
        Initializes the parameters for the Perceptron model. "features"
        is a list of all of the features of the model where each is
        represented by a string.
        """

        # NOTE: Do not change the names of these 3 data members because
        # they are used in the unit tests
        self.eta = 1
        self.theta = 0
        self.w = {feature: 0.0 for feature in features}

    def process_example(self, x, y):
        y_pred = self.predict_single(x)
        if y != y_pred:
            for feature, value in x.items():
                self.w[feature] += self.eta * y * value
            self.theta += self.eta * y

    def predict_single(self, x):
        score = 0
        for feature, value in x.items():
            score += self.w[feature] * value
        score += self.theta
        if score <= 0:
            return -1
        return 1
```

For the rest of the Perceptron-based algorithms, you will have to implement the corresponding class like we have done for `Perceptron`.
Use the `Perceptron` class as a guide for how to implement the functions.

### 1.3 Winnow (5 points)

#### 1.3.1 Description
The Winnow algorithm is a variant of the Perceptron algorithm with multiplicative updates. Since the algorithm requires that the target function is monotonic, you will only use it on the synthetic datasets.

The Winnow algorithm only learns parameters $\mathbf{w}$.
We will fix $\theta = -n$, where $n$ is the number of features.
When the Winnow algorithm makes a mistake on the example $(\textbf{x}, y)$, the parameters are updated with the following equation:
$$
\begin{equation}
    w^\textrm{new}_i \gets w_i \cdot \alpha^{y \cdot x_i}
\end{equation}
$$
where $w_i$ and $x_i$ are the $i$th components of the corresponding vectors.
Here, $\alpha$ is a promotion/demotion hyperparameter.

#### 1.3.2 Hyperparameters

For the experiment, choose $\alpha \in \{1.1, 1.01, 1.005, 1.0005, 1.0001\}$.

#### 1.3.3 Initialization

$\mathbf{w} = [1, 1, \dots, 1]$ and $\theta = -n$ (constant).

```python
class Winnow(Classifier):
    def __init__(self, alpha, features):
        # Do not change the names of these 3 data members because
        # they are used in the unit tests
        self.alpha = alpha
        self.w = {feature: 1.0 for feature in features}
        self.theta = -len(features)

    def process_example(self, x, y):
        y_pred = self.predict_single(x)
        if y != y_pred:
            for feature, value in x.items():
                self.w[feature] *= np.power(self.alpha, y * value)

    def predict_single(self, x):
        score = 0
        for feature, value in x.items():
            score += self.w[feature] * value
        score += self.theta
        if score <= 0:
            return -1
        return 1
```

### 1.4 AdaGrad (10 points)

#### 1.4.1 Description
AdaGrad is a variant of the Perceptron algorithm that adapts the learning rate for each parameter based on historical information.
    The idea is that frequently changing features get smaller learning rates and stable features higher ones.

To derive the update equations for this model, we first need to start with the loss function.
Instead of using the hinge loss with the elbow at 0 (like the basic Perceptron does), we will instead use the standard hinge loss with the elbow at 1:

$$
\begin{equation}
    \mathcal{L}(\mathbf{x}, y, \mathbf{w}, \theta) = \max\left\{0, 1 - y(\mathbf{w}^\intercal \mathbf{x} + \theta)\right\}
\end{equation}
$$

Then, by taking the partial derivative of $\mathcal{L}$ with respect to $\mathbf{w}$ and $\theta$, we can derive the respective graidents (make sure you understand how you could derive these gradients on your own):

$$
\begin{align}
    \frac{\partial\mathcal{L}}{\partial\mathbf{w}} &=
        \begin{cases}
        \mathbf{0} & \text{if $y(\mathbf{w}^\intercal \mathbf{x} + \theta) > 1$} \\
        -y\cdot \mathbf{x} & \textrm{otherwise}
        \end{cases} \\
    \frac{\partial\mathcal{L}}{\partial\theta} &=
        \begin{cases}
            0 & \text{if $y(\mathbf{w}^\intercal \mathbf{x} + \theta) > 1$} \\
            -y & \textrm{otherwise}
        \end{cases}
\end{align}
$$

Then for each parameter, we will keep track of the sum of the parameters' squared gradients.
In the following equations, the $k$ superscript refers to the $k$th non-zero gradient (i.e., the $k$th weight vector/misclassified example) and $t$ is the number of mistakes seen thus far.

$$
\begin{align}
    G^t_j &= \sum_{k=1}^t \left(\frac{\partial \mathcal{L}}{\partial w^k_j}\right)^2 \\
    H^t &= \sum_{k=1}^t \left(\frac{\partial \mathcal{L}}{\partial \theta^k}\right)^2
\end{align}
$$

For example, on the 3rd mistake ($t = 3$), $G^3_j$ is the sum of the squares of the first three non-zero gradients ($\left(\frac{\partial \mathcal{L}}{\partial w^1_j}\right)^2$, $\left(\frac{\partial \mathcal{L}}{\partial w^2_j}\right)^2$, and $\left(\frac{\partial \mathcal{L}}{\partial w^3_j}\right)^2$).
Then $\mathbf{G}^3$ is used to calculate the 4th value of the weight vector as follows.
On example $(\mathbf{x}, y)$, if $y(\mathbf{w}^\intercal \mathbf{x} + \theta) \leq 1$, then the parameters are updated with the following equations:

$$
\begin{align}
    \mathbf{w}^{t+1} &\gets \mathbf{w}^t + \eta \cdot \frac{y \cdot \mathbf{x}}{\sqrt{\mathbf{G}^t}} \\
    \theta^{t+1} &\gets \theta^t + \eta \frac{y}{\sqrt{H^t}}
\end{align}
$$

Note that, although we use the hinge loss with the elbow at 1 for training, you still make the prediction based on whether or not $y(\mathbf{w}^\intercal \mathbf{x} + \theta) \leq 0$ during testing.

#### 1.4.2 Hyperparameters

For the experiment, choose $\eta \in \{1.5, 0.25, 0.03, 0.005, 0.001\}$

#### 1.4.3 Initialization

$\mathbf{w} = [0, 0, \dots, 0]$ and $\theta = 0$.

```python
class AdaGrad(Classifier):
    def __init__(self, eta, features):
        self.eta = eta
        self.w = {feature: 0.0 for feature in features}
        self.theta = 0
        self.G = {feature: 1e-5 for feature in features}
        self.H = 0

    def process_example(self, x, y):
        score = 0
        for feature, value in x.items():
            score += self.w[feature] * value
        score += self.theta

        if y * score <= 1:
            for feature, value in x.items():
                self.G[feature] += np.power(-y * value, 2)
                self.w[feature] += self.eta * y * value / np.sqrt(self.G[feature])
            self.H += np.power(-y, 2)
            self.theta += self.eta * y / np.sqrt(self.H)

    def predict_single(self, x):
        score = 0
        for feature, value in x.items():
            score += self.w[feature] * value
        score += self.theta
        if score <= 0:
            return -1
        return 1
```

### 1.5 Averaged Models (15 points)

You will also implement the averaged version of the previous three algorithms.

During the course of training, each of the above algorithms will have $K + 1$ different parameter settings for the $K$ different updates it will make during training.
The regular implementation of these algorithms uses the parameter values after the $K$th update as the final ones.
Instead, the averaged version use the weighted average of the $K + 1$ parameter values as the final parameter values.
Let $m_k$ denote the number of correctly classified examples by the $k$th parameter values and $M$ the total number of correctly classified examples.
The final parameter values are

$$
\begin{align}
    M &= \sum_{k=1}^{K+1} m_k \\
    \mathbf{w} &\gets \frac{1}{M} \sum_{k=1}^{K+1} m_k \cdot \mathbf{w}^k \\
    \theta &\gets \frac{1}{M} \sum_{k=1}^{K+1} m_k \cdot \theta^k \\
\end{align}
$$

For each of the averaged versions of Perceptron, Winnow, and AdaGrad, use the same hyperparameters and initialization as before.

#### 1.5.1 Implementation Note
Implementing the averaged variants of these algorithms can be tricky.
    While the final parameter values are based on the sum of $K$ different vectors, there is no need to maintain *all* of these parameters.
    Instead, you should implement these algorithms by keeping only two vectors, one which maintains the cumulative sum and the current one.

Additionally, there are two ways of keeping track of these two vectors.
One is more straightforward but prohibitively slow.
The second requires some algebra to derive but is significantly faster to run.
Try to analyze how the final weight vector is a function of the intermediate updates and their corresponding weights.
It should take less than a minute or two for ten iterations for any of the averaged algorithms.
**You need to think about how to efficiently implement the averaged algorithms yourself.**

Further, the implementation for Winnow is slightly more complicated than the other two, so if you consistently have low accuracy for the averaged Winnow, take a closer look at the derivation.

```python
class AveragedPerceptron(Classifier):
    def __init__(self, features):
        self.eta = 1
        self.w = {feature: 0.0 for feature in features}
        self.w_sum = {feature: 0.0 for feature in features}
        self.theta = 0
        self.theta_sum = 0
        self.consistency_count = 1
        self.total = 1

    def process_example(self, x, y):
        y_pred = self.predict_single(x)
        if y != y_pred:
            for feature, value in x.items():
                self.w[feature] += self.eta * y * value
            self.theta += self.eta * y

            for feature, value in x.items():
                self.w_sum[feature] += self.eta * y * value * self.total
            self.theta_sum += self.eta * y * self.total

            self.total += self.consistency_count
            self.consistency_count = 1
        else:
            self.consistency_count += 1

    def finalize(self):
        self.total += self.consistency_count
        for feature, weight in self.w_sum.items():
            self.w[feature] -= weight / self.total
        self.theta -= self.theta_sum / self.total

    def predict_single(self, x):
        score = 0
        for feature, value in x.items():
            score += self.w[feature] * value
        score += self.theta
        if score <= 0:
            return -1
        return 1
```

```python
class AveragedWinnow(Classifier):
    def __init__(self, alpha, features):
        self.alpha = alpha
        self.features = features
        self.w = {feature: 1.0 for feature in features}
        self.w_sum = {feature: 0.0 for feature in features}
        self.theta = -len(features)
        self.consistency_count = 1
        self.total = 1

    def process_example(self, x, y):
        y_pred = self.predict_single(x)
        if y != y_pred:
            for feature, value in x.items():
                self.w_sum[feature] += self.w[feature] * (np.power(self.alpha, y * value) - 1) * self.total
                self.w[feature] *= np.power(self.alpha, y * value)
            self.total += self.consistency_count
            self.consistency_count = 1
        else:
            self.consistency_count += 1

    def finalize(self):
        self.total += self.consistency_count
        for feature in self.features:
            self.w[feature] = self.w[feature] - self.w_sum[feature] / self.total

    def predict_single(self, x):
        score = 0
        for feature, value in x.items():
            score += self.w[feature] * value
        score += self.theta
        if score <= 0:
            return -1
        return 1
```

```python
class AveragedAdaGrad(Classifier):
    def __init__(self, eta, features):
        self.eta = eta
        self.w = {feature: 0.0 for feature in features}
        self.w_sum = {feature: 0.0 for feature in features}
        self.theta = 0
        self.theta_sum = 0
        self.G = {feature: 1e-5 for feature in features}
        self.H = 0
        self.consistency_count = 1
        self.total = 1

    def process_example(self, x, y):
        score = 0
        for feature, value in x.items():
            score += self.w[feature] * value
        score += self.theta

        if y * score <= 1:
            for feature, value in x.items():
                self.G[feature] += np.power(-y * value, 2)
                self.w[feature] += self.eta * y * value / np.sqrt(self.G[feature])
            self.H += np.power(-y, 2)
            self.theta += self.eta * y / np.sqrt(self.H)

            for feature, value in x.items():
                self.w_sum[feature] += self.eta * y * value / np.sqrt(self.G[feature]) * self.total
            self.theta_sum += self.eta * y / np.sqrt(self.H) * self.total
            self.total += self.consistency_count
            self.consistency_count = 1
        else:
            self.consistency_count += 1

    def finalize(self):
        self.total += self.consistency_count
        for feature, weight in self.w_sum.items():
            self.w[feature] -= weight / self.total
        self.theta -= self.theta_sum / self.total

    def predict_single(self, x):
        score = 0
        for feature, value in x.items():
            score += self.w[feature] * value
        score += self.theta
        if score <= 0:
            return -1
        return 1
```

### 1.6 Support Vector Machines

Although we have not yet covered SVMs in class, you can still train them using the `sklearn` library.
We will use a soft margin SVM for non-linearly separable data.
You should use the `sklearn` implementation as follows:
```
from sklearn.svm import LinearSVC
classifier = LinearSVC(loss='hinge')
classifier.fit(X, y)
```

`sklearn` requires a different feature representation than what we use for the Perceptron models.
The provided Python template code demonstrates how to convert to the require representation.

Given training samples $S = \{(\mathbf{x}^1, y^1), (\mathbf{x}^2, y^2), \dots, (\mathbf{x}^m, y^m)\}$, the objective for the SVM is the following:

$$
\begin{equation}
    \min_{\mathbf{w}, b, \boldsymbol{\xi}} \frac{1}{2} \vert\vert \mathbf{w}\vert\vert^2_2 + C \sum_{i=1}^m \xi_i
\end{equation}
$$

subject to the following constraints:

$$
\begin{align}
    y^i(\mathbf{w}^\intercal \mathbf{x}^i + b) \geq 1 - \xi_i \;\;&\textrm{for } i = 1, 2, \dots, m \\
    \xi_i \geq 0 \;\;& \textrm{for } i = 1, 2, \dots, m
\end{align}
$$

```python
from sklearn.svm import LinearSVC
svm = LinearSVC(loss='hinge')
```

# Section 2. Datasets

In this section, we describe the synthetic and NER datasets that you will use for your experiments.
For the NER datasets, there is also an explanation for the features which you need to extract from the data.

### 2.1 Synthetic Data

#### 2.1.1 Introduction

The synthetic datasets have features and labels which are automatically generated from a python script.
Each instance will have $n$ binary features and are labeled according to a $l$-of-$m$-of-$n$ Boolean function.
Specifically, there is a set of $m$ features such that an example if positive if and only if at least $l$ of these $m$ features are active.
The set of $m$ features is the same for the dataset (i.e., it is not a separate set of $m$ features for each individual instance).

We provide two versions of the synthetic dataset called sparse and dense.
For both datasets, we set $l = 10$ and $m=20$.
We set $n = 200$ for the sparse data and $n = 40$ for the dense data.
Additionally, we add noise to the data as follows:
With probability $0.05$ the label assigned by the function is changed and with probability $0.001$ each feature value is changed.
Consequently, the data is not linearly separable.

We have provided you with three data splits for both sparse and dense with 10,000 training, 2,000 development, and 2,000 unlabeled testing examples (for submission).
Section 3 describes the experiments that you need to run on these datasets.

#### 2.1.2 Feature Representation

The features of the synthetic data provided are vectors of 0s and 1s.
Storing these large matrices requires lots of memory so we use a sparse representation that stores them as dictionaries instead.
For example, the vector $[0,1,0,0,0,1]$ can be stored as `{"x2": 1,"x6": 1}` (using 1-based indexing).
We have provided you with the code for parsing and converting the data to this format.
You can use these for the all algorithms you develop except the SVM.
Since you will be using the implementation of SVM from sklearn, you will need to provide a vector to it. You can use `sklearn.feature_extraction.DictVectorizer` for converting feature-value dictionaries to vectors.

### 2.2 NER Data

In addition to the synthetic data, we have provided you two datasets for the task of named-entity recognition (NER).
The goal is to identify whether strings in text represent names of people, organizations, or locations.
An example instance looks like the following:

```
    [PER Wolff] , currently a journalist in [LOC Argentina] , played with
    [PER del Bosque] in the final years of the seventies in
    [ORG Real Madrid] .
```

In this problem, we will simplify the task to identifying whether a string is named entity or not (that is, you don't have to say which type of entity it is).
For each token in the input, we will use the tag $\texttt{I}$ to denote that token is an entity and $\texttt{O}$ otherwise.
For example, the full tagging for the above instace is as follows:

```
    [I Wolff] [O ,] [O currently] [a] [O journalist] [O in] [I Argentina]
    [O ,] [O played] [O with] [I del] [I Bosque] [O in] [O the] [O final]
    [O years] [O of] [O the] [O seventies] [O in] [I Real] [I Madrid] .
```

Given a sentence $S = w_1, w_2, \dots, w_n$, you need to predict the `I`, `O` tag for each word in the sentence.
That is, you will produce the sequence $Y = y_1, y_2, \dots, y_n$ where $y_i \in$ {`I`, `O`}.

#### 2.2.1 Datasets: CoNLL and Enron

We have provided two datasets, the CoNLL dataset which is text from news articles, and Enron, a corpus of emails.
The files contain one word and one tag per line.
For CoNLL, there are training, development, and testing files, whereas Enron only has a test dataset.
There are 14,987 training sentences (204,567 words), 336 development sentences (3,779 words), and 303 testing sentences (3,880 words) in CoNLL.
For Enron there are 368 sentences (11,852 words).

**Please note that the CoNLL dataset is available only for the purposes of this assignment.
It is copyrighted, and you are granted access because you are a Penn student, but please delete it when you are done with the homework.**

#### 2.2.2 Feature Extraction

The NER data is provided as raw text, and you are required to extract features for the classifier.
In this assignment, we will only consider binary features based on the context of the word that is supposed to be tagged.

Assume that there are $V$ unique words in the dataset and each word has been assigned a unique ID which is a number $\{1, 2, \dots, V\}$.
Further, $w_{-k}$ and $w_{+k}$ indicate the $k$th word before and after the target word.
The feature templates that you should use to generate features are as follows:

| Template             | Number of Features |
|----------------------|--------------------|
| $w_{-3}$             | $V$                |
| $w_{-2}$             | $V$                |
| $w_{-1}$             | $V$                |
| $w_{+1}$             | $V$                |
| $w_{+2}$             | $V$                |
| $w_{+3}$             | $V$                |
| $w_{-1}$ & $w_{-2}$  | $V \times V$       |
| $w_{+1}$ \& $w_{+2}$ | $V \times V$       |
| $w_{-1}$ \& $w_{+1}$ | $V \times V$       |

Each feature template corresponds to a set of features that you will compute (similar to the features you generated in problem 2 from the first homework assignment).
The $w_{-3}$ feature template corresponds to $V$ features where the $i$th feature is 1 if the third word to the left of the target word has ID $i$.
The $w_{-1} \& w_{+1}$ feature template corresponds to $V \times V$ features where there is one feature for every unique pair words.
For example, feature $(i - 1) \times V + j$ is a binary feature that is 1 if the word 1 to the left of the target has ID $i$ and the first word to the right of the target has ID $j$.
In practice, you will not need to keep track of the feature IDs.
Instead, each feature will be given a name such as "$w_{-1}=\textrm{the} \& w_{+1}=\textrm{cat}$".

In total, all of the above feature templates correspond to a very large number of features.
However, for each word, there will be exactly 9 features which are active (non-zero), so the feature vector is quite sparse.
You will represent this as a dictionary which maps from the feature name to the value.
In the provided Python template, we have implemented a couple of the features for you to demonstrate how to compute them and what the naming scheme should look like.

In order to deal with the first two words and the last two words in a sentence, we will add special symbol "SSS" and "EEE" to the vocabulary to represent the words before the first word and the words after the last word.
Notice that in the test data you may encounter a word that was not observed in training, and therefore is not in your dictionary.
In this case, you cannot generate a feature for it, resulting in less than 7 active features in some of the test examples.

# Section 3. Experiments (65 points total)

You will run two sets of experiments, one using the synthetic data and one using the NER data.

### 3.1 Synthetic Experiment (30 + 10 extra credit points)

This experiment will explore the impact that the amount of training data has on model performance.
First, you will do hyperparameter tuning for Winnow and Perceptron with AdaGrad (both standard and averaged versions).
Then you will generate learning curves that will plot the size of the training data against the performance.
Finally, for each of the models trained on all of the training data, you will run it on unlabeled test examples for leader board submission.
You should use accuracy to compute the performance of the model.

In summary, the experiment consists of three parts
1. Parameter Tuning
2. Learning Curves
3. Final Evaluation

#### 3.1.1 Parameter Tuning (10 points)

For both the Winnow and Perceptron with AdaGrad (standard and averaged), there are hyperparameters that you need to choose.
(The same is true for SVM, but you should only use the default settings.)
Similarly to cross-validation from Homework 1, we will estimate how well each model will do on the true test data using the development dataset (we will not run cross-validation), and choose the hyperparameter settings based on these results.

For each hyperparameter value in Section 1, train a model using that value on the training data and compute the accuracy on the development dataset. Each model should be trained for 10 iterations (i.e., 10 passes over the entire dataset).

##### Winnow Sweep

| $\alpha$ | Sparse | Dense  |
|----------|--------|--------|
| 1.1      | 0.8935 | 0.8995 |
| 1.01     | 0.9270 | 0.9215 |
| 1.005    | 0.9195 | 0.9080 |
| 1.0005   | 0.5630 | 0.8615 |
| 1.0001   | 0.5205 | 0.6140 |

##### Averaged Winnow Sweep

| $\alpha$ | Sparse | Dense  |
|----------|--------|--------|
| 1.1      | 0.9390 | 0.9445 |
| 1.01     | 0.8875 | 0.9310 |
| 1.005    | 0.8145 | 0.9130 |
| 1.0005   | 0.5257 | 0.6820 |
| 1.0001   | 0.5055 | 0.5465 |

##### AdaGrad Sweep

| $\eta$   | Sparse | Dense  |
|----------|--------|--------|
| 1.5      | 0.9130 | 0.9445 |
| 0.25     | 0.7695 | 0.7825 |
| 0.03     | 0.6635 | 0.5795 |
| 0.005    | 0.5000 | 0.5085 |
| 0.001    | 0.4985 | 0.5085 |

##### Averaged AdaGrad Sweep

| $\eta$   | Sparse | Dense  |
|----------|--------|--------|
| 1.5      | 0.8525 | 0.9445 |
| 0.25     | 0.7620 | 0.7055 |
| 0.03     | 0.6430 | 0.5885 |
| 0.005    | 0.5055 | 0.5085 |
| 0.001    | 0.5590 | 0.5085 |

#### 3.1.2 Learning Curves (10 points)

Next, you will train all 7 models with different amounts of training data.
For Winnow and Perceptron with AdaGrad (standard and averaged), use the best hyperparameters from the parameter tuning experiment.

Each of the datasets contains 10,000 training examples.
You will train each model 11 times on varying amounts of training data.
The first 10 will increase by 500 examples: 500, 1k, 1.5k, 2k, ..., 5k.
The 11th model should use all 10k examples.
Each Perceptron-based model should be trained for 10 iterations (e.g., 10 passes over the total number of training examples available to that model).
The SVM can be run until convergence with the default parameters.

For each model, compute the accuracy on the development dataset and plot the results using the provided code.
There should be a separate plot for the sparse and dense data.

**Note** how we have included an image in markdown. You should do the same for both plots and include them in the output below by running your experiment, saving your plots to the images folder, and linking it to this cell.

##### Sparse Plot

##### Dense Plot

#### 3.1.3 Final Evaluation (5 points)

Finally, for each of the 7 models, train the models on all of the training data and compute the test accuracy.
For Winnow and Perceptron with AdaGrad, use the best hyperparameter settings you found.
Report these accuracies in a table.

We will run our models with [500, 1000, 1500, 2000, 2500, 3000, 3500, 4000, 4500, 5000, 10000] examples.

```python
def run_synthetic_experiment(dataset_type='sparse'):
    """
    Runs the synthetic experiment on either the sparse or dense data
    depending on the data path (e.g. "data/sparse" or "data/dense").

    We have provided how to train the Perceptron on the training and
    test on the testing data (the last part of the experiment). You need
    to implement the hyperparameter sweep, the learning curves, and
    predicting on the test dataset for the other models.
    """

    X_train, y_train, X_dev, y_dev, X_test, y_test, features = load_synthetic_data(dataset_type)

    def hyperparameter_sweep():
        models_with_hyperparameters = [
            ((Winnow, AveragedWinnow), 'alpha', [1.1, 1.01, 1.005, 1.0005, 1.0001]),
            ((AdaGrad, AveragedAdaGrad), 'eta', [1.5, 0.25, 0.03, 0.005, 0.001]),
        ]
        best_hyperparams = {}
        for models, hp_name, hyperparameters in models_with_hyperparameters:
            for model in models:
                best_acc, best_hyperparameter = 0, None
                for hyperparameter in hyperparameters:
                    classifier = model(hyperparameter, features)
                    classifier.train(X_train, y_train)
                    y_pred = classifier.predict(X_dev)
                    acc = accuracy_score(y_dev, y_pred)
                    if acc > best_acc:
                        best_acc = acc
                        best_hyperparameter = hyperparameter
                    print(f'For {model.__name__}, {hp_name}={hyperparameter} => accuracy={acc}')
                print(f"For {model.__name__} the best accuracy is {best_acc} " +
                      f"with hyperparameter {best_hyperparameter}")
                best_hyperparams[model.__name__] = best_hyperparameter
        return best_hyperparams

    def learning_curves(best_hyperparams):
        # Each model along the best hyperparameter
        sample_sizes = [500 * i for i in range(1, 11)] + [10_000]
        models_accuracies = {
            model_name: [] for model_name in
            [
                'Perceptron',
                'Winnow',
                'AdaGrad',
                'AveragedPerceptron',
                'AveragedWinnow',
                'AveragedAdaGrad',
                'SVM'
            ]
        }
        # Include the two last models not used in the previous part.
        best_hyperparams.update({
            'Perceptron': None,
            'AveragedPerceptron': None
        })
        for sample_size in sample_sizes:
            for model, best_hyper in best_hyperparams.items():
                model = getattr(sys.modules[__name__], model)
                classifier = model(features) if not best_hyper else model(best_hyper, features)
                classifier.train(X_train[:sample_size], y_train[:sample_size])
                y_pred = classifier.predict(X_dev)
                acc = accuracy_score(y_dev, y_pred)
                models_accuracies[model.__name__].append(acc)

            # SVM is a bit different.
            vectorizer = DictVectorizer()
            X_train_dict = vectorizer.fit_transform(X_train[:sample_size])
            X_dev_dict = vectorizer.transform(X_dev)
            svm.fit(X_train_dict, y_train[:sample_size])
            y_pred = svm.predict(X_dev_dict)
            acc = accuracy_score(y_dev, y_pred)
            models_accuracies['SVM'].append(acc)
        plot_learning_curves(
            models_accuracies['Perceptron'],
            models_accuracies['Winnow'],
            models_accuracies['AdaGrad'],
            models_accuracies['AveragedPerceptron'],
            models_accuracies['AveragedWinnow'],
            models_accuracies['AveragedAdaGrad'],
            models_accuracies['SVM']
        )

    def predict_test_dataset(best_hyperparams):

        # Include the two last models not used in the previous part.
        best_hyperparams.update({
            'Perceptron': None,
            'AveragedPerceptron': None
        })

        accuracies = {}
        for model, best_hyper in best_hyperparams.items():
            model = getattr(sys.modules[__name__], model)
            classifier = model(features) if not best_hyper else model(best_hyper, features)
            classifier.train(X_train, y_train)
            y_pred = classifier.predict(X_test)
            acc = accuracy_score(y_test, y_pred)
            accuracies[model.__name__] = acc

        # SVM is a bit different.
        vectorizer = DictVectorizer()
        X_train_dict = vectorizer.fit_transform(X_train)
        X_test_dict = vectorizer.transform(X_test)
        classifier = LinearSVC(loss='hinge')
        classifier.fit(X_train_dict, y_train)
        y_pred = classifier.predict(X_test_dict)
        acc = accuracy_score(y_test, y_pred)
        accuracies['SVM'] = acc

        print(accuracies)

    # Experiment
    best_hyperparams = hyperparameter_sweep()
    learning_curves(best_hyperparams)
    predict_test_dataset(best_hyperparams)
```

```python
%%time
"""
Run the synthetic experiment on the sparse dataset. For reference,
"synthetic/sparse" is the path to where the data is located.
"""
run_synthetic_experiment('sparse')
```

```python
%%time
"""
Run the synthetic experiment on the dense dataset. For reference,
"synthetic/dense" is the path to where the data is located.
"""
run_synthetic_experiment('dense')
```

##### Questions (5 points)

Answer the following questions:

1. Discuss the trends that you see when comparing the standard version of an algorithm to the averaged version (e.g., Winnow versus Averaged Winnow). Is there an observable trend?

    In general, it does appear that the averaged versions of the models learn quicker and have better overall performance than the standard implementations.

2. We provided you with 10,000 training examples.
   Were all 10,000 necessary to achieve the best performance for each classifier?
   If not, how many were necessary? (Rough estimates, no exact numbers required)

    For the dense dataset, it appears that the majority of the models' performances plateaued around 2,000 training examples, so the additional 8,000 did not help. However, for the sparse dataset, it looks like the model performance did not plateau within the amount of training data that we ran the learning curve for, and so the models did require more than 5,000 training examples to learn from.

3. Report your Final Test Accuracies

| Model               | Sparse | Dense  |
|---------------------|--------|--------|
| Perceptron          | 0.7170 | 0.9205 |
| Winnow              | 0.9260 | 0.9255 |
| AdaGrad             | 0.9120 | 0.9405 |
| Averaged Perceptron | 0.9090 | 0.9405 |
| Averaged Winnow     | 0.9360 | 0.9405 |
| Averaged AdaGrad    | 0.8370 | 0.9405 |
| SVM                 | 0.9360 | 0.9405 |

#### 3.1.5 Extra Credit (10 points)

Included in the resources for this homework assignment is the code that we used to generate the synthetic data.
We used a small amount of noise to create the dataset which you ran the experiments on.
For extra credit, vary the amount of noise in either/both of the label and features.
Then, plot the models' performances as a function of the amount of noise.
Discuss your observations.

In general, we looked for experiments which increased the amount of noise and compared model performance for each different level.
    As the amount of noise increased, the model performance should have dropped because the true learning signal becomes weaker and weaker.

### 3.2 NER Experiment: Welcome to the Real World (35 points)

The experiment with the NER data will analyze how changing the domain of the training and testing data can impact the performance of a model.

Instead of accuracy, you will use your $F_1$ score implementation in Section 0 to evaluate how well a model does.
Recall measures how many of the actual entities the model successfully tagged as an entity.

$$
\begin{align}
    \textrm{Precision} &= \frac{\#\textrm{(Actually Entity & Model Predicted Entity)}}{\#\textrm{(Model Predicted Entity)}} \\
    \textrm{Recall} &= \frac{\#\textrm{(Actually Entity & Model Predicted Entity)}}{\#\textrm{(Actually Entity)}} \\
    \textrm{F}_1 &= 2 \cdot \frac{\textrm{Precision} \times \textrm{Recall}}{\textrm{Precision} + \textrm{Recall}}
\end{align}
$$

For this experiment, you will only use the averaged basic Perceptron and SVM.
Hence, no parameter tuning is necessary.
Train both models on the CoNLL training data then compute the F$_1$ on the development and testing data of both CoNLL and Enron.
Note that the model which is used to predict labels for Enron is trained on CoNLL data, not Enron data.
Report the F$_1$ scores in a table.

#### 3.2.1 Extracting NER Features  (25 points)

Reread Section 2.2.2 to understand how to extract the features required to train the models
and translate it to the code below.

```python
def extract_ner_features_train(train):
    """
    Extracts feature dictionaries and labels from the data in "train"
    Additionally creates a list of all of the features which were created.
    We have implemented the w-1 and w+1 features for you to show you how
    to create them.

    TODO: You should add your additional featurization code here.
    (which might require adding and/or changing existing code)
    """
    y = []
    X = []
    features = set()
    for sentence in train:
        padded = [('SSS', None)] + sentence[:] + [('EEE', None)]
        for i in range(1, len(padded) - 1):
            y.append(1 if padded[i][1] == 'I' else -1)
            feat1 = 'w-1=' + str(padded[i - 1][0])
            feat2 = 'w+1=' + str(padded[i + 1][0])
            feats = [feat1, feat2]
            features.update(feats)
            feats = {feature: 1 for feature in feats}
            X.append(feats)
    return features, X, y
```

Now, repeat the process of extracting features from the test data.
What is the difference between the code above and below?

```python
def extract_features_dev_or_test(data, features):
    """
    Extracts feature dictionaries and labels from "data". The only
    features which should be computed are those in "features". You
    should add your additional featurization code here.

    TODO: You should add your additional featurization code here.
    """
    y = []
    X = []
    for sentence in data:
        padded = [('SSS', None)] + sentence[:] + [('EEE', None)]
        for i in range(1, len(padded) - 1):
            y.append(1 if padded[i][1] == 'I' else -1)
            feat1 = 'w-1=' + str(padded[i - 1][0])
            feat2 = 'w+1=' + str(padded[i + 1][0])
            feats = [feat1, feat2]
            feats = {feature: 1 for feature in feats if feature in features}
            X.append(feats)
    return X, y
```

#### 3.2.2 Running the NER Experiment

As stated previously, train both models on the CoNLL training data then compute the $F_1$  on the development and testing data of both CoNLL and Enron. Note that the model which is used to predict labels for Enron is trained on CoNLL data, not Enron data.

```python
def run_ner_experiment(data_path):
    """
    Runs the NER experiment using the path to the ner data
    (e.g. "ner" from the released resources). We have implemented
    the standard Perceptron below. You should do the same for
    the averaged version and the SVM.

    The SVM requires transforming the features into a different
    format. See the end of this function for how to do that.
    """
    train = load_ner_data(dataset='conll', dataset_type='train')
    conll_test = load_ner_data(dataset='conll', dataset_type='test')
    enron_test = load_ner_data(dataset='enron', dataset_type='test')

    features, X_train, y_train = extract_ner_features_train(train)
    X_conll_test, y_conll_test = extract_features_dev_or_test(conll_test, features)
    X_enron_test, y_enron_test = extract_features_dev_or_test(enron_test, features)

    # TODO: We show you how to do this for Perceptron.
    # You should do this for the Averaged Perceptron and SVM
    classifier = Perceptron(features)
    classifier.train(X_train, y_train)
    y_pred = classifier.predict(X_conll_test)

    conll_f1 = calculate_f1(y_conll_test, y_pred)
    y_pred = classifier.predict(X_enron_test)
    generate_txt_predictions(y_pred, 'NER_Perceptron_pred')

    enron_f1 = calculate_f1(y_enron_test, y_pred)
    print('Averaged Perceptron')
    print('  CoNLL', conll_f1)
    print('  Enron', enron_f1)

    # This is how you convert from the way we represent features in the
    # Perceptron code to how you need to represent features for the SVM.
    # You can then train with (X_train_dict, y_train) and test with
    # (X_conll_test_dict, y_conll_test) and (X_enron_test_dict, y_enron_test)
    vectorizer = DictVectorizer()
    X_train_dict = vectorizer.fit_transform(X_train)
    X_conll_test_dict = vectorizer.transform(X_conll_test)
    X_enron_test_dict = vectorizer.transform(X_enron_test)
```

```python
# Run the NER experiment. "ner" is the path to where the data is located.
run_ner_experiment('ner')
```

##### F1 Scores Table
TODO: report your values:

| Model               | CoNLL Test F1 | Enron Test F1 |
|---------------------|---------------|---------------|
| Averaged Perceptron |               |               |
| SVM                 |               |               |

##### Questions (5 points)

Comment on the results:
1. Are the F$_1$ scores on CoNLL and Enron similar?

    The F$_1$ scores on the Enron test set are significantly lower than on the CoNLL test set. This is an indication that the models trained on CoNLL do not generalize to out-of-domain data, a problem known as domain transfer.

2. If they are dissimilar, explain why you think the F$_1$ score increased/decreased on the Enron data.

    The most likely explanation is that because the text of emails is so different from news corpora, the features learned by the model fire less frequently and there is signal in the emails which is not present in news text, so the features were not learned. For example, the CoNLL model may have learned that "New", "York", and "Times" are all good indications of an entity. However, in email communications, people may write "new york times" or "nyt" instead. If they do, the features learned from CoNLL do not fire (because "New York Times" isn't present with the capitalized text) and positives features from Enron like "new", "york", and "times" were never observed in CoNLL, so the model did not learn a positive weight for them.

## Submission Instructions

We will be using Gradescope to turn in both the Python code.
You should have been automatically added to Gradescope.
If you do not have access, please ask the TA staff on Piazza.

There are four parts to the submission on Gradescope:
* `.ipynb` file: Submit this notebook (.ipynb). If you are using Google Colab, you will have to download it as an ipynb file.
* PDF: This notebook saved as a PDF. We will use this to see the overall structure of your homework and code, and to check manual questions like the tables, plots, and your discussions. How should I convert this notebook to PDF, you may ask? There are a few ways, but for simplicity print the Jupyter notebook and _Save as PDF_.
* Code: A `hw2.py` file. We will use this to unit test and automate grading some parts of the homework. We only need a few functions/classes from the whole notebook. In particular, to unit test we only need:
    - `calculate_f1`
    - `highest_and_lowest_f1_score`
    - `Perceptron`, `Winnow`, and `AdaGrad` classes.

There are two ways to submit these pieces of code. You can either manually copy and paste these to a Python file,
or you can use Jupyter's _Download as .py_, and delete all unnecessary code.