# SNLP Assignment 4

Name 1: Nikhil Paliwal

Student id 1: 7009915

Email 1: nipa00002@stud.uni-saarland.de

Name 2: Sangeet Sagar

Student id 2: 7009050

Email 2: sasa00001@stud.uni-saarland.de

**Instructions:** Read each question carefully.

Make sure you appropriately comment your code wherever required. Your final submission should contain the completed Notebook and the Python file for exercise 2.

Upload the zipped folder in Teams. Make sure to click on "Turn-in" after you upload your submission, otherwise the assignment will not be considered as submitted. Only one member of the group should make the submisssion.

---

## Exercise 1: Encodings (3 points)

#### 1.1 (0.5 points)

What are the benefits of having a fixed-width code as opposed to variable-width? e.g. `0011` for `e` and `1111001010` for `Z` (since `e` is much more common than `Z`, this would save some space in variable length encoding).

Imagine that you are given a string in UTF16 (variable-width character encoding). What issues can you encounter? Provide a specific engineering example.

#### 1.2 (1 point)

The ASCII encoding uses 8 bits per single character. The target alphabet in this case is binary: $\{0, 1\}$. Given a text of length $n$, the encoding takes $n\times 8$ bits in memory (without taking alignment into consideration).

- How would you adapt the ASCII encoding if the target alphabet had three symbols $\{0,1,2\}$?
- How many "trits" (number of symbols from  {0,1,2}) would it take to encode $n$ characters in this new system? Be precise in your mathematical explanation, and be careful which way you round and beware of $\pm 1$ errors.
- The UTF32 encoding uses 32 bits per single character. How does your computation change?

#### 1.3 (1.5 points)

Given the following text, construct a [Huffman tree](https://en.wikipedia.org/wiki/Huffman_coding) and encode it (at character level).

- Show the final tree (ideally as a diagram together with frequency counts). You can work it out on a paper and include a photograph.
- Show the encoded sequence (ideally in comparison to ASCII encoded sequence in binary).
- Compare the length to ASCII encoding (n*8 characters). What improvement did you achieve? Could you expect the same amount of improvement on the entirety of Wikipedia?

```
Three thousand three hundred and thirty-three silver syringes.
```

Treat space as a character as well and `t` and `T` as two different characters.

## Exercise 2: Conditional Entropy of DNA (7 points)
In this exercise, we will see how conditional entropy is calculated for genome sequences. Read the instructions given below carefully.

### 2.0 Getting started with biopython

1. **Installing biopython**

Install [biopython](https://biopython.org/) to your local Python environment. Dependencies should be installed automatically.

The installation instructions for biopython are given in the attached link. A simple pip installation does the trick.
2. **Saving data files**

Download the genome of *Drosophila melanogaster* from [kaggle](https://www.kaggle.com/mylesoneill/drosophila-melanogaster-genome?select=genome.fa). For this, you wil need to create an account on kaggle.

To save the data files you download, create a folder called `data` and save the csv and fasta files directly in this folder, since the same path will be used for retrieving the relevant files, as you can see below in the provided code. Conversely, you can simply rename the `archive` folder as `data`.
3. **How to read a fasta file**

As your first task, you have to sample a reduced version of the genome using the pre-implemented `sample_records` function in `exercise_2.py`. The function will write the reduced genome to a [fasta](https://en.wikipedia.org/wiki/FASTA_format) file.

Biopython can be imported in your Python file with the statement `import Bio`.
For reading fasta files, look up how to utilise biopython's SeqIO module for parsing the file structure to read genome information in the form of sequences. The result should be akin to reading a list of sentences from a text file. Check the `sample_records` function for further guidance.
4. **Understanding genome sequences**

Parsing the fasta file results in an [iterator](https://wiki.python.org/moin/Iterator) of sequence records for the genomes in a class called SeqRecord.

Structure: *SeqRecord*

Elements:

o Meta-features of the sequence (*Name*, *Id*, *Description*, *Number of features*)

o The actual sequence of nucleotides (*Seq*)

These sequences comprise chains formed of the same basic building blocks - the nucleotides A, G, C, T. You will observe that these sequences are case sensitive i.e. they contain both uppercase and lowercase chains. We will explicitly tell you what they mean and how to handle them over the course of this exercise.

5. **Processing genome sequences**

We can deal with the obtained sequences as if they were normal Python strings. Your goal is to first combine all these sequences into a continuous string of nucleotides, and then extract k-mers from this final "text" corpus.
5. **Difference between k-mers and n-grams**

For all intents and purposes, k-mers are to genome sequences what n-grams are to word sequences.

```python
# 2.0 Sample a reduced version of the genome
from importlib import reload
from pathlib import Path

import exercise_2
exercise_2 = reload(exercise_2)

N = 100

genome_loc = Path("../data/genome.fa")
genome_red_loc = Path("../data/genome_reduced.fa")

exercise_2.sample_records(genome_loc, genome_red_loc, N)
```

### 2.1: K-mers (0.5 points)

[k-mers](https://en.wikipedia.org/wiki/K-mer) are sequences of nucleotides of length $k$. You can see them as the DNA variant of n-grams, only with nucleotides as its "words".

Implement the function `get_k_mers` that assembles sequences of nucleotides from the fasta file produced in 2.0. For now, convert all characters to uppercase.

e. g. for a sequence
```
GTAGAGCTGT
```
The 2-mers to be sampled are, just as in a bigram language model:
```
GT, TA, AG, GA, AG, GC, CT, TG, GT
```
This example is taken from the Wikipedia article, see there for higher $k$.

Now, show the output of the function for $k = 2$, i. e. all 2-mers.

```python
# 2.1 get k-mers
#  REMOVE LIMITS
k_mers = exercise_2.get_k_mers(genome_red_loc, 2)
print(k_mers)
```

    , 'GA', 'AT', 'TC', 'CC', 'CC', 'CT', 'TT', 'TC', 'CG', 'GA', 'AA', 'AC', 'CT', 'TT', 'TG', 'GT', 'TT', 'TT', 'TT', 'TA', 'AC', 'CG', 'GT', 'TG', 'GG', 'GC', 'CG', 'GT', 'TG', 'GT', 'TT', 'TC', 'CC', 'CA', 'AC', 'CA', 'AC', 'CT', 'TG', 'GA', 'AA', 'AG', 'GG', 'GG', 'GA', 'AT', 'TT', 'TA', 'AC', 'CA', 'AA', 'AC', 'CC', 'CA', 'AC', 'CG', 'GG', 'GC', 'CA', 'AC', 'CC', 'CC', 'CT', 'TA', 'AT', 'TA', 'AC', 'CA', 'AC', 'CC', 'CC', 'CA', 'AC', 'CA', 'AA', 'AG', 'GT', 'TG', 'GA', 'AG', 'GC', 'CA', 'AT', 'TA', 'AT', 'TG', 'GG', 'GT', 'TG', 'GC', 'CA', 'AT', 'TG', 'GT', 'TG', 'GT', 'TC', 'CC', 'CG', 'GT', 'TA', 'AA', 'AC', 'CT', 'TC', 'CA', 'AA', 'AA', 'AC', 'CG', 'GT', 'TC', 'CG', 'GA', 'AG', 'GC', 'CA', 'AT', 'TG', 'GA', 'AC', 'CC', 'CC', 'CG', 'GC', 'CG', 'GA', 'AT', 'TA', 'AT', 'TG', 'GG', 'GG', 'GT', 'TA', 'AC', 'CC', 'CA', 'AA', 'AC', 'CC', 'CA', 'AC', 'CA', 'AG', 'GC', 'CC', 'CA', 'AC', 'CC', 'CA', 'AC', 'CG', 'GA', 'AT', 'TA', 'AC', 'CT', 'TC', 'CT', 'TC', 'CA', 'AC', 'CC', 'CA', 'AG', 'GT', 'TG', 'GG', 'GG', 'GC', 'CC', 'CT', 'TA', 'AC', 'CC', 'CT', 'TC', 'CA', 'AA', 'AT', 'TG', 'GA', 'AG', 'GC', 'CA', 'AG', 'GT', 'TC', 'CT', 'TT', 'TG', 'GG', 'GA', 'AT', 'TA', 'AG', 'GC', 'CT', 'TG', 'GC', 'CT', 'TC', 'CA', 'AA', 'AC', 'CC', 'CC', 'CG', 'GT', 'TG', 'GG', 'GT', 'TA', 'AC', 'CC', 'CG', 'GA', 'AA', 'AA', 'AA', 'AG', 'GA', 'AG', 'GG', 'GC', 'CT', 'TC', 'CG', 'GG', 'GT', 'TG', 'GG', 'GG', 'GC', 'CC', 'CT', 'TC', 'CC', 'CC', 'CC', 'CC', 'CT', 'TT', 'TT', 'TT', 'TG', 'GC', 'CT', 'TC', 'CC', 'CG', 'GA', 'AC', 'CA', 'AA', 'AG', 'GC', 'CA', 'AT', 'TG', 'GG', 'GT', 'TT', 'TA', 'AG', 'GA', 'AG', 'GG', 'GA', 'AA', 'AC', 'CC', 'CT', 'TA', 'AT', 'TC', 'CG', 'GC', 'CT', 'TG', 'GT', 'TA', 'AT', 'TT', 'TA', 'AG', 'GT', 'TC', 'CG', 'GT', 'TC', 'CT', 'TC', 'CT', 'TT', 'TA', 'AC', 'CG', 'GA', 'AC', 'CA', 'AA', 'AG', 'GC', 'CA', 'AA', 'AC', 'CA', 'AG', 'GC', 'CG', 'GG', 'GA', 'AC', 'CT', 'TG', 'GT', 'TG', 'GG', 'GT', 'TA', 'AG', 'GT', 'TA', 'AT', 'TT', 'TC', 'CT', 'TC', 'CG', 'GC', 'CC', 'CC', 'CG', 'GC', 'CT', 'TA', 'AA', 'AC', 'CC', 'CA', 'AC', 'CA', 'AC', 'CG', 'GG', 'GC', 'CA', 'AA', 'AA', 'AG', 'GA', 'AT', 'TG', 'GT', 'TT', 'TG', 'GA', 'AG', 'GC', 'CT', 'TA', 'AA', 'AA', 'AC', 'CG', 'GC', 'CC', 'CC', 'CC', 'CT', 'TC', 'CC', 'CT', 'TC', 'CC', 'CT', 'TC', 'CA', 'AG', 'GG', 'GA', 'AA', 'AC', 'CA', 'AC', 'CT', 'TT', 'TC', 'CC', 'CT', 'TC', 'CC', 'CG', 'GC', 'CG', 'GA', 'AA', 'AC', 'CA', 'AT', 'TT', 'TG', 'GC', 'CC', 'CA', 'AG', 'GC', 'CC', 'CG', 'GT', 'TT', 'TG', 'GA', 'AG', 'GT', 'TC', 'CT', 'TT', 'TC', 'CT', 'TC', 'CT', 'TT', 'TG', 'GA', 'AT', 'TC', 'CA', 'AT', 'TC', 'CC', 'CA', 'AG', 'GG', 'GA', 'AT', 'TT', 'TC', 'CT', 'TC', 'CT', 'TC', 'CG', 'GA', 'AA', 'AG', 'GA', 'AT', 'TC', 'CC', 'CA', 'AG', 'GT', 'TT', 'TT', 'TT', 'TC', 'CG', 'GC', 'CC', 'CA', 'AA', 'AG', 'GG', 'GC', 'CG', 'GC', 'CT', 'TG', 'GC', 'CA', 'AC', 'CT', 'TC', 'CC', 'CA', 'AA', 'AG', 'GT', 'TC', 'CC', 'CA', 'AT', 'TC', 'CT', 'TA', 'AG', 'GG', 'GT', 'TC', 'CC', 'CC', 'CG', 'GC', 'CA', 'AG', 'GA', 'AT', 'TC', 'CT', 'TG', 'GC', 'CA', 'AT', 'TA', 'AT', 'TA', 'AG', 'GG', 'GG', 'GG', 'GG', 'GC', 'CA', 'AA', 'AG', 'GC', 'CG', 'GC', 'CA', 'AC', 'CA', 'AA', 'AA', 'AA', 'AT', 'TG', 'GT', 'TG', 'GC', 'CA', 'AT', 'TC', 'CC', 'CA', 'AA', 'AT', 'TC', 'CC', 'CT', 'TC', 'CA', 'AT', 'TA', 'AT', 'TG', 'GG', 'GA', 'AT', 'TC', 'CT', 'TC', 'CC', 'CA', 'AC', 'CA', 'AT', 'TG', 'GC', 'CT', 'TC', 'CA', 'AC', 'CG', 'GG', 'GA', 'AG', 'GT', 'TG', 'GG', 'GT', 'TA', 'AT', 'TC', 'CG', 'GC', 'CT', 'TG', 'GA', 'AG', 'GG', 'GG', 'GC', 'CT', 'TC', 'CT', 'TC', 'CC', 'CC', 'CG', 'GT', 'TG', 'GC', 'CA', 'AA', 'AA', 'AA', 'AA', 'AT', 'TG', 'GG', 'GC', 'CA', 'AT', 'TT', 'TA', 'AA', 'AA', 'AC', 'CG', 'GA', 'AC', 'CC', 'CC', 'CG', 'GT', 'TG', 'GC', 'CC', 'CG', 'GT', 'TG', 'GT', 'TA', 'AA', 'AG', 'GC', 'CA', 'AG', 'GG', 'GA', 'AA', 'AA', 'AG', 'GA', 'AC', 'CG', 'GT', 'TC', 'CC', 'CT', 'TC', 'CA', 'AT', 'TC', 'CG', 'GA', 'AG', 'GA', 'AA', 'AT', 'TC', 'CC', 'CA', 'AA', 'AA', 'AA', 'AC', 'CT', 'TT', 'TG', 'GA', 'AA', 'AT', 'TC', 'CC', 'CC', 'CG', 'GA', 'AT', 'TA', 'AG', 'GG', 'GC', 'CA', 'AA', 'AG', 'GA', 'AG', 'GT', 'TG', 'GA', 'AC', 'CG', 'GT', 'TA', 'AT', 'TG', 'GG', 'GG', 'GA', 'AT', 'TA', 'AA', 'AA', 'AC', 'CC', 'CT', 'TA', 'AT', 'TG', 'GC', 'CG', 'GT', 'TC', 'CA', 'AC', 'CC', 'CC', 'CG', 'GT', 'TC', 'CC', 'CT', 'TG', 'GG', 'GT', 'TT', 'TC', 'CG', 'GC', 'CT', 'TG', 'GT', 'TC', 'CC', 'CT', 'TC', 'CG', 'GT', 'TC', 'CC', 'CC', 'CA', 'AT', 'TC', 'CT', 'TG', 'GT', 'TT', 'TC', 'CT', 'TG', 'GC', 'CC', 'CA', 'AT', 'TA', 'AA', 'AT', 'TA', 'AA', 'AC', 'CG', 'GT', 'TA', 'AA', 'AC', 'CT', 'TT', 'TA', 'AT', 'TG', 'GG', 'GC', 'CT', 'TG', 'GC', 'CT', 'TA', 'AA', 'AT', 'TA', 'AA', 'AC', 'CT', 'TT', 'TA', 'AG', 'GC', 'CA', 'AG', 'GC', 'CC', 'CA', 'AG', 'GA', 'AT', 'TC', 'CA', 'AA', 'AG', 'GC', 'CG', 'GG', 'GG', 'GG', 'GG', 'GA', 'AG', 'GC', 'CT', 'TC', 'CC', 'CA', 'AC', 'CC', 'CA', 'AA', 'AG', 'GT', 'TA', 'AC', 'CC', 'CT', 'TG', 'GC', 'CA', 'AG', 'GT', 'TG', 'GC', 'CC', 'CA', 'AC', 'CT', 'TG', 'GT', 'TG', 'GG', 'GA', 'AC', 'CA', 'AC', 'CT', 'TG', 'GT', 'TT', 'TC', 'CG', 'GG', 'GC', 'CA', 'AT', 'TA', 'AC', 'CC', 'CG', 'GA', 'AA', 'AA', 'AG', 'GG', 'GC', 'CA', 'AT', 'TC', 'CC', 'CA', 'AA', 'AG', 'GC', 'CA', 'AG', 'GG', 'GA', 'AT', 'TT', 'TA', 'AG', 'GC', 'CC', 'CT', 'TC', 'CT', 'TG', 'GG', 'GC', 'CA', 'AG', 'GG', 'GA', 'AG', 'GG', 'GC', 'CT', 'TA', 'AG', 'GT', 'TC', 'CG', 'GC', 'CC', 'CT', 'TC', 'CT', 'TG', 'GG', 'GG', 'GC', 'CA', 'AG', 'GC', 'CT', 'TA', 'AC', 'CT', 'TT', 'TG', 'GT', 'TT', 'TC', 'CC', 'CG', 'GC', 'CG', 'GG', 'GT', 'TG', 'GT', 'TC', 'CA', 'AT', 'TA', 'AC', 'CC', 'CA', 'AT', 'TA', 'AC', 'CC', 'CG', 'GG', 'GG', 'GG', 'GC', 'CA', 'AC', 'CC', 'CA', 'AA', 'AA', 'AC', 'CA', 'AG', 'GC', 'CA', 'AC', 'CA', 'AC', 'CA', 'AA', 'AG', 'GG', 'GT', 'TG', 'GC', 'CC', 'CA', 'AT', 'TG', 'GA', 'AG', 'GT', 'TC', 'CC', 'CG', 'GT', 'TC', 'CA', 'AT', 'TA', 'AT', 'TA', 'AT', 'TG', 'GG', 'GT', 'TC', 'CC', 'CG', 'GC', 'CC', 'CT', 'TG', 'GG', 'GC', 'CT', 'TC', 'CG', 'GA', 'AG', 'GG', 'GA', 'AC', 'CT', 'TG', 'GA', 'AA', 'AG', 'GC', 'CC', 'CC', 'CC', 'CA', 'AG', 'GT', 'TC', 'CG', 'GG', 'GC', 'CT', 'TC', 'CG', 'GA', 'AA', 'AG', 'GC', 'CA', 'AC', 'CA', 'AC', 'CG', 'GC', 'CG', 'GC', 'CC', 'CA', 'AA', 'AT', 'TG', 'GC', 'CT', 'TC', 'CC', 'CA', 'AA', 'AC', 'CG', 'GA', 'AC', 'CT', 'TC', 'CC', 'CG', 'GG', 'GT', 'TC', 'CA', 'AT', 'TC', 'CC', 'CG', 'GC', 'CT', 'TG', 'GG', 'GC', 'CG', 'GA', 'AA', 'AG', 'GC', 'CG', 'GA', 'AA', 'AG', 'GC', 'CT', 'TA', 'AT', 'TG', 'GT', 'TG', 'GC', 'CG', 'GT', 'TG', 'GA', 'AG', 'GG', 'GA', 'AA', 'AT', 'TT', 'TT', 'TC', 'CA', 'AT', 'TT', 'TC', 'CC', 'CT', 'TT', 'TC', 'CA', 'AC', 'CT', 'TG', 'GA', 'AC', 'CC', 'CG', 'GT', 'TG', 'GA', 'AT', 'TG', 'GC', 'CC', 'CA', 'AA', 'AG', 'GG', 'GT', 'TA', 'AC', 'CC', 'CG', 'GA', 'AC', 'CA', 'AG', 'GC', 'CT', 'TA', 'AC', 'CG', 'GC', 'CA', 'AC', 'CA', 'AT', 'TA', 'AC', 'CG', 'GG', 'GA', 'AA', 'AG', 'GG', 'GT', 'TT', 'TC', 'CG', 'GC', 'CT', 'TC', 'CC', 'CA', 'AG', 'GC', 'CA', 'AA', 'AA', 'AC', 'CC', 'CT', 'TC', 'CA', 'AC', 'CC', 'CG', 'GA', 'AA', 'AG', 'GG', 'GC', 'CG', 'GC', 'CA', 'AC', 'CG', 'GT', 'TC', 'CT', 'TC', 'CA', 'AA', 'AG', 'GG', 'GC', 'CA', 'AC', 'CC', 'CT', 'TT', 'TT', 'TC', 'CA', 'AG', 'GC', 'CA', 'AG', 'GC', 'CA', 'AT', 'TG', 'GA', 'AT', 'TT', 'TA', 'AC', 'CC', 'CG', 'GT', 'TC', 'CT', 'TT', 'TG', 'GC', 'CT', 'TG', 'GG', 'GT', 'TC', 'CG', 'GA', 'AG', 'GA', 'AC', 'CG', 'GG', 'GC', 'CA', 'AA', 'AC', 'CG', 'GC', 'CC', 'CA', 'AA', 'AC', 'CT', 'TT', 'TC', 'CC', 'CG', 'GC', 'CT', 'TC', 'CC', 'CC', 'CC', 'CA', 'AC', 'CG', 'GT', 'TT', 'TT', 'TC', 'CT', 'TA', 'AC', 'CG', 'GA', 'AT', 'TG', 'GG', 'GA', 'AA', 'AA', 'AC', 'CC', 'CT', 'TC', 'CC', 'CT', 'TT', 'TT', 'TT', 'TT', 'TC', 'CC', 'CT', 'TC', 'CT', 'TA', 'AG', 'GC', 'CA', 'AC', 'CA', 'AG', 'GC', 'CT', 'TC', 'CG', 'GG', 'GG', 'GA', 'AA', 'AT', 'TT', 'TT', 'TC', 'CC', 'CC', 'CT', 'TC', 'CG', 'GA', 'AC', 'CG', 'GA', 'AG', 'GA', 'AA', 'AG', 'GC', 'CA', 'AG', 'GC', 'CA', 'AA', 'AG', 'GT', 'TC', 'CA', 'AT', 'TC', 'CC', 'CG', 'GC', 'CG', 'GT', 'TA', 'AT', 'TG', 'GC', 'CA', 'AC', 'CT', 'TC', 'CA', 'AG', 'GC', 'CT', 'TG', 'GG', 'GC', 'CA', 'AA', 'AT', 'TA', 'AC', 'CG', 'GG', 'GC', 'CT', 'TG', 'GG', 'GA', 'AG', 'GA', 'AC', 'CG', 'GC', 'CT', 'TG', 'GA', 'AA', 'AG', 'GC', 'CA', 'AG', 'GT', 'TA', 'AC', 'CA', 'AT', 'TC', 'CC', 'CA', 'AT', 'TC', 'CA', 'AG', 'GT', 'TA', 'AT', 'TG', 'GT', 'TC', 'CC', 'CC', 'CA', 'AG', 'GA', 'AT', 'TA', 'AA', 'AA', 'AT', 'TG', 'GG', 'GG', 'GC', 'CC', 'CG', 'GC', 'CT', 'TG', 'GA', 'AT', 'TT', 'TG', 'GA', 'AT', 'TC', 'CC', 'CC', 'CT', 'TG', 'GC', 'CG', 'GG', 'GG', 'GC', 'CA', 'AG', 'GC', 'CC', 'CT', 'TC', 'CT', 'TA', 'AG', 'GT', 'TT', 'TA', 'AC', 'CC', 'CG', 'GG', 'GT', 'TA', 'AC', 'CA', 'AT', 'TC', 'CC', 'CA', 'AC', 'CA', 'AG', 'GT', 'TA', 'AA', 'AC', 'CG', 'GA', 'AA', 'AA', 'AC', 'CT', 'TG', 'GC', 'CT', 'TT', 'TC', 'CG', 'GG', 'GA', 'AT', 'TC', 'CA', 'AC', 'CT', 'TG', 'GC', 'CT', 'TC', 'CT', 'TT', 'TC', 'CG', 'GG', 'GC', 'CC', 'CG', 'GG', 'GA', 'AG', 'GA', 'AA', 'AA', 'AA', 'AA', 'AG', 'GC', 'CT', 'TC', 'CT', 'TG', 'GC', 'CC', 'CA', 'AC', 'CA', 'AA', 'AG', 'GC', 'CC', 'CC', 'CA', 'AT', 'TT', 'TT', 'TC', 'CC', 'CC', 'CG', 'GG', 'GC', 'CA', 'AT', 'TC', 'CC', 'CC', 'CA', 'AA', 'AG', 'GT', 'TC', 'CG', 'GG', 'GC', 'CT', 'TA', 'AG', 'GT', 'TC', 'CG', 'GG', 'GC', 'CA', 'AG', 'GA', 'AG', 'GT', 'TG', 'GC', 'CA', 'AA', 'AG', 'GC', 'CA', 'AC', 'CT', 'TC', 'CC', 'CA', 'AT', 'TT', 'TC', 'CG', 'GA', 'AC', 'CG', 'GT', 'TT', 'TG', 'GT', 'TC', 'CG', 'GA', 'AA', 'AT', 'TG', 'GC', 'CT', 'TC', 'CC', 'CT', 'TT', 'TT', 'TG', 'GA', 'AA', 'AG', 'GT', 'TC', 'CC', 'CA', 'AC', 'CG', 'GA', 'AA', 'AT', 'TG', 'GT', 'TG', 'GC', 'CC', 'CG', 'GA', 'AG', 'GC', 'CA', 'AC', 'CG', 'GT', 'TA', 'AT', 'TT', 'TG', 'GC', 'CG', 'GC', 'CC', 'CG', 'GG', 'GG', 'GC', 'CT', 'TG', 'GG', 'GC', 'CG', 'GC', 'CC', 'CA', 'AA', 'AC', 'CA', 'AC', 'CT', 'TG', 'GC', 'CC', 'CT', 'TC', 'CC', 'CA', 'AA', 'AG', 'GC', 'CA', 'AT', 'TC', 'CC', 'CT', 'TC', 'CC', 'CA', 'AC', 'CA', 'AC', 'CA', 'AT', 'TC', 'CG', 'GT', 'TC', 'CC', 'CT', 'TT', 'TG', 'GG', 'GC', 'CG', 'GA', 'AA', 'AA', 'AT', 'TC', 'CC', 'CA', 'AA', 'AA', 'AT', 'TT', 'TG', 'GC', 'CC', 'CA', 'AT', 'TC', 'CT', 'TG', 'GC', 'CA', 'AG', 'GC', 'CC', 'CT', 'TT', 'TC', 'CC', 'CG', 'GG', 'GA', 'AA', 'AG', 'GA', 'AA', 'AC', 'CT', 'TT', 'TC', 'CT', 'TC', 'CT', 'TC', 'CA', 'AC', 'CA', 'AC', 'CG', 'GA', 'AT', 'TT', 'TC', 'CA', 'AT', 'TG', 'GA', 'AT', 'TG', 'GG', 'GC', 'CC', 'CT', 'TC', 'CG', 'GA', 'AG', 'GC', 'CA', 'AC', 'CC', 'CT', 'TT', 'TA', 'AC', 'CC', 'CA', 'AA', 'AA', 'AG', 'GA', 'AC', 'CT', 'TG', 'GG', 'GT', 'TA', 'AG', 'GC', 'CA', 'AA', 'AG', 'GC', 'CA', 'AT', 'TA', 'AT', 'TT', 'TC', 'CC', 'CT', 'TC', 'CT', 'TA', 'AT', 'TA', 'AC', 'CG', 'GA', 'AG', 'GG', 'GA', 'AG', 'GG', 'GG', 'GC', 'CT', 'TC', 'CA', 'AC', 'CA', 'AC', 'CT', 'TT', 'TG', 'GT', 'TC', 'CC', 'CT', 'TT', 'TA', 'AT', 'TC', 'CT', 'TG', 'GG', 'GC', 'CC', 'CC', 'CT', 'TT', 'TT', 'TG', 'GA', 'AG', 'GC', 'CA', 'AG', 'GC', 'CG', 'GA', 'AG', 'GA', 'AC', 'CG', 'GA', 'AC', 'CT', 'TC', 'CG', 'GT', 'TG', 'GG', 'GG', 'GC', 'CA', 'AC', 'CT', 'TT', 'TC', 'CC', 'CA', 'AC', 'CT', 'TC', 'CG', 'GG', 'GC', 'CT', 'TG', 'GG', 'GA', 'AA', 'AA', 'AG', 'GT', 'TA', 'AT', 'TC', 'CC', 'CC', 'CA', 'AA', 'AT', 'TC', 'CG', 'GG', 'GA', 'AT', 'TG', 'GC', 'CA', 'AT', 'TC', 'CG', 'GG', 'GG', 'GA', 'AA', 'AA', 'AA', 'AC', 'CA', 'AA', 'AT', 'TG', 'GA', 'AT', 'TG', 'GC', 'CT', 'TA', 'AG', 'GG', 'GT', 'TG', 'GC', 'CT', 'TC', 'CA', 'AG', 'GG', 'GT', 'TA', 'AT', 'TG', 'GG', 'GC', 'CG', 'GC', 'CG', 'GC', 'CC', 'CA', 'AG', 'GA', 'AC', 'CT', 'TG', 'GC', 'CC', 'CT', 'TT', 'TG', 'GC', 'CA', 'AG', 'GA', 'AT', 'TA', 'AG', 'GT', 'TG', 'GC', 'CC', 'CA', 'AT', 'TT', 'TG', 'GA', 'AT', 'TG', 'GC', 'CC', 'CG', 'GT', 'TC', 'CC', 'CA', 'AA', 'AC', 'CC', 'CC', 'CG', 'GG', 'GG', 'GA', 'AG', 'GA', 'AG', 'GC', 'CG', 'GC', 'CC', 'CT', 'TG', 'GC', 'CT', 'TC', 'CT', 'TT', 'TC', 'CA', 'AA', 'AC', 'CC', 'CG', 'GG', 'GG', 'GC', 'CA', 'AA', 'AC', 'CA', 'AC', 'CA', 'AT', 'TG', 'GC', 'CA', 'AT', 'TC', 'CA', 'AA', 'AC', 'CC', 'CT', 'TC', 'CG', 'GA', 'AA', 'AT', 'TA', 'AC', 'CT', 'TT', 'TC', 'CG', 'GA', 'AG', 'GG', 'GG', 'GC', 'CC', 'CG', 'GG', 'GT', 'TG', 'GC', 'CT', 'TT', 'TC', 'CG', 'GG', 'GA', 'AC', 'CT', 'TC', 'CC', 'CG', 'GC', 'CA', 'AA', 'AC', 'CT', 'TG', 'GG', 'GG', 'GA', 'AA', 'AG', 'GA', 'AA', 'AA', 'AT', 'TT', 'TG', 'GC', 'CG', 'GG', 'GA', 'AG', 'GG', 'GA', 'AG', 'GC', 'CA', 'AC', 'CT', 'TC', 'CG', 'GT', 'TG', 'GC', 'CG', 'GC', 'CA', 'AG', 'GT', 'TC', 'CA', 'AC', 'CC', 'CC', 'CC', 'CA', 'AA', 'AT', 'TC', 'CA', 'AG', 'GT', 'TG', 'GA', 'AT', 'TC', 'CA', 'AG', 'GC', 'CT', 'TC', 'CG', 'GC', 'CC', 'CT', 'TC', 'CG', 'GG', 'GC', 'CA', 'AA', 'AA', 'AT', 'TC', 'CT', 'TT', 'TG', 'GT', 'TA', 'AG', 'GA', 'AC', 'CG', 'GC', 'CG', 'GC', 'CC', 'CC', 'CC', 'CC', 'CA', 'AT', 'TG', 'GG', 'GG', 'GT', 'TC', 'CG', 'GT', 'TC', 'CG', 'GC', 'CC', 'CA', 'AT', 'TG', 'GA', 'AT', 'TC', 'CT', 'TC', 'CC', 'CC', 'CA', 'AC', 'CG', 'GA', 'AA', 'AG', 'GC', 'CG', 'GT', 'TT', 'TT', 'TC', 'CC', 'CA', 'AG', 'GT', 'TC', 'CA', 'AT', 'TC', 'CC', 'CA', 'AT', 'TC', 'CT', 'TT', 'TG', 'GG', 'GC', 'CC', 'CC', 'CT', 'TC', 'CC', 'CC', 'CA', 'AA', 'AT', 'TG', 'GA', 'AG', 'GC', 'CT', 'TT', 'TC', 'CT', 'TT', 'TG', 'GT', 'TA', 'AG', 'GT', 'TT', 'TG', 'GG', 'GC', 'CT', 'TG', 'GA', 'AG', 'GG', 'GC', 'CA', 'AC', 'CG', 'GC', 'CC', 'CT', 'TC', 'CA', 'AG', 'GC', 'CT', 'TC', 'CG', 'GA', 'AC', 'CC', 'CA', 'AC', 'CT', 'TA', 'AC', 'CA', 'AA', 'AG', 'GC', 'CT', 'TC', 'CT', 'TA', 'AC', 'CA', 'AG', 'GC', 'CG', 'GC', 'CC', 'CA', 'AC', 'CT', 'TT', 'TA', 'AG', 'GA', 'AA', 'AA', 'AT', 'TG', 'GA', 'AA', 'AT', 'TT', 'TT', 'TG', 'GT', 'TA', 'AT', 'TC', 'CC', 'CC', 'CA', 'AC', 'CT', 'TT', 'TC', 'CT', 'TG', 'GA', 'AA', 'AT', 'TT', 'TG', 'GT', 'TC', 'CA', 'AG', 'GG', 'GT', 'TT', 'TG', 'GA', 'AT', 'TC', 'CT', 'TT', 'TC', 'CA', 'AT', 'TG', 'GT', 'TT', 'TA', 'AA', 'AT', 'TT', 'TC', 'CT', 'TT', 'TT', 'TT', 'TC', 'CT', 'TT', 'TT', 'TT', 'TA', 'AT', 'TT', 'TA', 'AA', 'AT', 'TG', 'GT', 'TG', 'GT', 'TT', 'TC', 'CC', 'CA', 'AA', 'AT', 'TA', 'AG', 'GT', 'TT', 'TA', 'AT', 'TA', 'AA', 'AG', 'GT', 'TC', 'CT', 'TT', 'TC', 'CA', 'AG', 'GT', 'TA', 'AT', 'TA', 'AC', 'CA', 'AT', 'TC', 'CT', 'TG', 'GC', 'CC', 'CT', 'TT', 'TC', 'CA', 'AT', 'TC', 'CG', 'GT', 'TT', 'TC', 'CA', 'AC', 'CG', 'GC', 'CA', 'AG', 'GC', 'CG', 'GA', 'AG', 'GC', 'CA', 'AC', 'CA', 'AC', 'CG', 'GT', 'TC', 'CC', 'CG', 'GC', 'CA', 'AC', 'CT', 'TC', 'CC', 'CC', 'CT', 'TG', 'GG', 'GC', 'CA', 'AT', 'TC', 'CT', 'TC', 'CG', 'GC', 'CA', 'AA', 'AG', 'GC', 'CA', 'AA', 'AC', 'CT', 'TC', 'CC', 'CC', 'CC', 'CC', 'CT', 'TC', 'CT', 'TC', 'CT', 'TC', 'CA', 'AT', 'TG', 'GC', 'CT', 'TC', 'CT', 'TT', 'TC', 'CC', 'CT', 'TC', 'CT', 'TT', 'TG', 'GA', 'AC', 'CA', 'AA', 'AT', 'TA', 'AG', 'GG', 'GA', 'AA', 'AT', 'TG', 'GT', 'TA', 'AT', 'TT', 'TT', 'TA', 'AT', 'TT', 'TA', 'AC', 'CA', 'AT', 'TT', 'TT', 'TA', 'AA', 'AC', 'CT', 'TT', 'TA', 'AG', 'GC', 'CC', 'CC', 'CA', 'AT', 'TT', 'TG', 'GA', 'AA', 'AT', 'TG', 'GG', 'GC', 'CG', 'GT', 'TA', 'AC', 'CT', 'TT', 'TC', 'CA', 'AC', 'CA', 'AT', 'TT', 'TT', 'TA', 'AT', 'TG', 'GA', 'AT', 'TT', 'TT', 'TC', 'CT', 'TT', 'TC', 'CT', 'TT', 'TT', 'TG', 'GA', 'AC', 'CA', 'AC', 'CG', 'GC', 'CC', 'CT', 'TG', 'GA', 'AT', 'TT', 'TT', 'TG', 'GA', 'AT', 'TT', 'TT', 'TT', 'TG', 'GC', 'CA', 'AA', 'AG', 'GT', 'TT', 'TA', 'AA', 'AA', 'AA', 'AG', 'GT', 'TT', 'TA', 'AT', 'TT', 'TT', 'TC', 'CT', 'TG', 'GT', 'TG', 'GT', 'TT', 'TA', 'AA', 'AC', 'CA', 'AT', 'TC', 'CT', 'TT', 'TT', 'TT', 'TC', 'CT', 'TG', 'GA', 'AT', 'TA', 'AG', 'GA', 'AG', 'GT', 'TT', 'TT', 'TC', 'CA', 'AT', 'TA', 'AG', 'GA', 'AG', 'GA', 'AA', 'AT', 'TT', 'TA', 'AG', 'GT', 'TA', 'AA', 'AA', 'AT', 'TT', 'TT', 'TA', 'AA', 'AT', 'TA', 'AG', 'GT', 'TC', 'CT', 'TT', 'TT', 'TC', 'CT', 'TT', 'TT', 'TA', 'AA', 'AA', 'AA', 'AG', 'GA', 'AC', 'CT', 'TA', 'AA', 'AA', 'AA', 'AG', 'GA', 'AC', 'CT', 'TG', 'GA', 'AT', 'TT', 'TG', 'GT', 'TT', 'TA', 'AA', 'AT', 'TT', 'TA', 'AA', 'AA', 'AA', 'AT', 'TT', 'TT', 'TG', 'GT', 'TT', 'TT', 'TG', 'GT', 'TT', 'TG', 'GC', 'CT', 'TA', 'AT', 'TT', 'TG', 'GT', 'TT', 'TA', 'AA', 'AT', 'TT', 'TA', 'AA', 'AA', 'AT', 'TC', 'CG', 'GT', 'TT', 'TC', 'CA', 'AT', 'TA', 'AT', 'TT', 'TA', 'AT', 'TC', 'CT', 'TT', 'TG', 'GA', 'AA', 'AG', 'GT', 'TA', 'AA', 'AA', 'AA', 'AT', 'TT', 'TA', 'AA', 'AG', 'GT', 'TC', 'CG', 'GT', 'TC', 'CG', 'GT', 'TG', 'GG', 'GT', 'TC', 'CT', 'TG', 'GG', 'GA', 'AG', 'GT', 'TA', 'AC', 'CC', 'CT', 'TG', 'GC', 'CA', 'AA', 'AT', 'TG', 'GT', 'TA', 'AC', 'CT', 'TT', 'TC', 'CC', 'CA', 'AT', 'TG', 'GC', 'CA', 'AG', 'GT', 'TT', 'TC', 'CC', 'CT', 'TA', 'AT', 'TT', 'TA', 'AA', 'AA', 'AT', 'TT', 'TG', 'GA', 'AT', 'TT', 'TG', 'GC', 'CT', 'TC', 'CT', 'TA', 'AG', 'GT', 'TG', 'GT', 'TT', 'TT', 'TC', 'CT', 'TA', 'AT', 'TT', 'TA', 'AT', 'TG', 'GT', 'TA', 'AG', 'GT', 'TG', 'GC', 'CA', 'AG', 'GG', 'GG', 'GG', 'GA', 'AT', 'TA', 'AA', 'AT', 'TG', 'GT', 'TT', 'TT', 'TG', 'GA', 'AA', 'AG', 'GA', 'AA', 'AT', 'TA', 'AG', 'GA', 'AG', 'GA', 'AA', 'AT', 'TG', 'GT', 'TT', 'TT', 'TG', 'GA', 'AA', 'AC', 'CC', 'CA', 'AA', 'AT', 'TT', 'TC', 'CG', 'GT', 'TG', 'GT', 'TT', 'TT', 'TT', 'TA', 'AA', 'AA', 'AG', 'GT', 'TG', 'GT', 'TG', 'GG', 'GT', 'TA', 'AA', 'AT', 'TT', 'TG', 'GT', 'TT', 'TT', 'TG', 'GA', 'AA', 'AT', 'TG', 'GT', 'TT', 'TT', 'TT', 'TA', 'AG', 'GT', 'TT', 'TC', 'CC', 'CG', 'GT', 'TT', 'TA', 'AT', 'TC', 'CT', 'TG', 'GC', 'CG', 'GT', 'TA', 'AT', 'TG', 'GA', 'AA', 'AG', 'GC', 'CT', 'TA', 'AA', 'AG', 'GG', 'GT', 'TA', 'AT', 'TT', 'TT', 'TA', 'AA', 'AA', 'AG', 'GT', 'TG', 'GT', 'TT', 'TT', 'TG', 'GT', 'TA', 'AA', 'AT', 'TG', 'GT', 'TG', 'GG', 'GT', 'TT', 'TT', 'TA', 'AT', 'TA', 'AT', 'TC', 'CT', 'TA', 'AT', 'TA', 'AA', 'AC', 'CA', 'AT', 'TA', 'AA', 'AA', 'AT', 'TT', 'TA', 'AA', 'AT', 'TC', 'CT', 'TT', 'TG', 'GT', 'TG', 'GG', 'GT', 'TG', 'GG', 'GA', 'AT', 'TG', 'GC', 'CT', 'TT', 'TG', 'GA', 'AA', 'AT', 'TT', 'TT', 'TT', 'TT', 'TC', 'CC', 'CT', 'TA', 'AA', 'AT', 'TC', 'CC', 'CT', 'TG', 'GT', 'TA', 'AT', 'TT', 'TT', 'TA', 'AT', 'TT', 'TA', 'AC', 'CT', 'TG', 'GC', 'CA', 'AG', 'GT', 'TC', 'CT', 'TG', 'GT', 'TA', 'AC', 'CG', 'GT', 'TC', 'CG', 'GG', 'GT', 'TA', 'AT', 'TA', 'AG', 'GT', 'TT', 'TG', 'GG', 'GT', 'TA', 'AA', 'AT', 'TG', 'GT', 'TC', 'CT', 'TA', 'AG', 'GA', 'AA', 'AG', 'GT', 'TC', 'CC', 'CA', 'AT', 'TG', 'GT', 'TA', 'AC', 'CT', 'TA', 'AT', 'TT', 'TG', 'GT', 'TT', 'TA', 'AT', 'TG', 'GG', 'GT', 'TG', 'GC', 'CA', 'AT', 'TA', 'AG', 'GG', 'GA', 'AT', 'TT', 'TA', 'AA', 'AT', 'TG', 'GC', 'CC', 'CC', 'CT', 'TA', 'AA', 'AA', 'AA', 'AA', 'AA', 'AA', 'AT', 'TA', 'AA', 'AA', 'AA', 'AT', 'TA', 'AA', 'AA', 'AT', 'TT', 'TT', 'TA', 'AA', 'AT', 'TT', 'TA', 'AT', 'TT', 'TT', 'TC', 'CG', 'GG', 'GA', 'AT', 'TG', 'GT', 'TT', 'TA', 'AC', 'CT', 'TT', 'TT', 'TT', 'TG', 'GT', 'TG', 'GT', 'TA', 'AT', 'TT', 'TT', 'TT', 'TC', 'CT', 'TT', 'TT', 'TC', 'CC', 'CT', 'TG', 'GA', 'AT', 'TT', 'TC', 'CT', 'TG', 'GT', 'TT', 'TA', 'AT', 'TG', 'GA', 'AC', 'CT', 'TG', 'GT', 'TA', 'AG', 'GT', 'TA', 'AT', 'TT', 'TC', 'CC', 'CT', 'TG', 'GT', 'TT', 'TT', 'TT', 'TC', 'CT', 'TT', 'TT', 'TA', 'AA', 'AC', 'CT', 'TA', 'AT', 'TT', 'TT', 'TC', 'CT', 'TT', 'TT', 'TT', 'TT', 'TT', 'TA', 'AT', 'TA', 'AT', 'TC', 'CT', 'TC', 'CG', 'GG', 'GC', 'CT', 'TG', 'GA', 'AT', 'TA', 'AA', 'AT', 'TT', 'TT', 'TT', 'TG', 'GA', 'AA', 'AC', 'CC', 'CA', 'AA', 'AG', 'GT', 'TC', 'CG', 'GG', 'GT', 'TG', 'GT', 'TG', 'GT', 'TT', 'TT', 'TA', 'AT', 'TC', 'CT', 'TT', 'TA', 'AA', 'AC', 'CA', 'AA', 'AT', 'TA', 'AA', 'AA', 'AT', 'TT', 'TT', 'TC', 'CG', 'GT', 'TC', 'CA', 'AC', 'CC', 'CA', 'AA', 'AT', 'TG', 'GT', 'TT', 'TA', 'AT', 'TA', 'AG', 'GT', 'TT', 'TT', 'TT', 'TT', 'TC', 'CA', 'AA', 'AA', 'AT', 'TG', 'GT', 'TG', 'GT', 'TT', 'TA', 'AC', 'CG', 'GT', 'TT', 'TT', 'TC', 'CT', 'TG', 'GG', 'GT', 'TT', 'TA', 'AT', 'TG', 'GG', 'GT', 'TA', 'AT', 'TT', 'TC', 'CG', 'GA', 'AG', 'GG', 'GT', 'TA', 'AT', 'TT', 'TT', 'TT', 'TT', 'TT', 'TC', 'CT', 'TG', 'GT', 'TT', 'TT', 'TG', 'GT', 'TC', 'CT', 'TA', 'AT', 'TA', 'AT', 'TT', 'TG', 'GT', 'TC', 'CA', 'AA', 'AG', 'GT', 'TC', 'CT', 'TA', 'AG', 'GT', 'TT', 'TT', 'TG', 'GT', 'TT', 'TC', 'CG', 'GT', 'TA', 'AA', 'AG', 'GC', 'CT', 'TT', 'TC', 'CT', 'TA', 'AA', 'AG', 'GC', 'CT', 'TA', 'AG', 'GG', 'GC', 'CA', 'AG', 'GT', 'TT', 'TA', 'AG', 'GA']

### 2.2 K-mer language models (3 points)

*  Implement the function `k_mer_statistics`. It should estimate $K$ language models with k-mer sizes $1,...,K$, and return two probability distributions:

a) relative k-mer frequencies and

b) conditional k-mer probabilities.

As always, you may modify the function signature to suit your needs (but you must comment on the changes you make). (2.5 points)

 **Hint:** Calculate the relative frequencies up to K, and then use these for calculating the $k^{th}$ level conditional probabilities using the $(k-1)^{th}$ level relative frequencies. (Refer to the formulae you obtained in Assignment 2 Ex. 2.1)
* Plot the probabilites of all k-mers for $K=5$ language models vs their rank, and use the log-log scale on the axes. Do so by implementing the function `plot_k_mers`. Does the curve look similar to the one you obtained for natural language? (0.5 points)

```python
# 2.2 k-mer statistics
rel_freqs, cond_probs = exercise_2.k_mer_statistics(genome_red_loc, K=5)

# plot
exercise_2.plot_k_mers(rel_freqs)
```

### 2.3 Conditional entropy (1.5 point)
We want to observe how the conditional entropy of our small DNA corpus changes with increasing k-mer size. Conditional Entropy is defined as

\begin{equation}
H(W|H) = - \sum_{i} p(w_i,h_i) \cdot \log_2p(w_i|h_i)
\end{equation}

Where $p(w_i,h_i)$ is the relative frequency of $(w_i, h_i)$ and $p(w_i|h_i)$ is the conditional probability of $w_i$ given the history $h_i$.

* What do $w_i$, $h_i$ and the probabilities derived from them correspond to in the context of DNA? (0.5 Points)

* Using your insights from above, implement the function `conditional_entropy`, that calculates the conditional entropy of a k-mer language model. You may modify the function signature as you wish. (0.5 points)

* Estimate up to $K=20$ k-mer language models, calculate their conditional entropies and plot by ascending $k$. What do you observe? (0.5 points)

### Answer:
1. $w_i$ refers to current nucleotide, $h_i$ refers to past sequence of nucleotides and the conditional probability refers to probability of occurance of current nucleotide given past sequence of nucleotides.
3. We observe that with increasing number of k, entropy decrease thus we get less information.

```python
# 2.3 conditional entropy
K = 20

H_ks = []

rel_freqs, cond_probs = exercise_2.k_mer_statistics(genome_red_loc, K=20)

for k in range(1, K+1):
  H_k = exercise_2.conditional_entropy(rel_freqs[k], cond_probs[k])
  print("{}-mer cond. entropy is {}".format(k, H_k))
  H_ks.append(H_k)

# plot
exercise_2.plot_conditional_entropies(H_ks)
```

### 2.4 Tandem repeats (2 points)

A [tandem repeat](https://en.wikipedia.org/wiki/Tandem_repeat) or a mini/microsattelite is a sequence of nucleotides that is repeated multiple times, and the repetitions are immediately adjacent. This example is again taken from the Wikipedia article:

```
ATTCG ATTCG ATTCG
```

In the *Drosophila Melanogaster* genome, tandem repeats are represented by lowercase letters, while the non-repeating sequences are in uppercase. Up to 2.3, we ignored tandem repeats and considered all sequences in uppercase.

* Read up about tandem repeats, and tell what your expectations about the conditional entropy of tandem repeat regions are as opposed to non-tandem repeat regions (0.5 points)

* Implement the functions `get_k_mers_24` and `k_mer_statistics_24` and sample the same language models as in 2.2, but this time exclusively on tandem repeats or non-tandem repeats. This should yield two sets of language models $LM_{TR}$ and $LM_{\neg TR}$. (1 point)

* Calculate the conditional entropies of both $LM_{TR}$ and $LM_{\neg TR}$. Plot the conditional entropies vs. increasing $K$ as in 2.3. Do you observe any difference? Does it follow your expectation from above? (0.5 points)

```python
# 2.4.1 tandem repeats
H_ks = []

rel_freqs, cond_probs = exercise_2.k_mer_statistics_24(genome_red_loc, K, tandem_repeats=True)

for k in range(K):
  H_k = exercise_2.conditional_entropy(rel_freqs[k], cond_probs[k])
  H_ks.append(H_k)

# plot
exercise_2.plot_conditional_entropies(H_ks)

# 2.4.2 non tandem repeats
H_ks = []

rel_freqs, cond_probs = exercise_2.k_mer_statistics_24(genome_red_loc, K, tandem_repeats=False)

for k in range(K):
  H_k = exercise_2.conditional_entropy(rel_freqs[k], cond_probs[k])
  H_ks.append(H_k)

# plot
exercise_2.plot_conditional_entropies(H_ks)
```

## Bonus (1.5 points)

The standard Huffman encoding uses a binary target alphabet $\{0,1\}$. Assume that you're given a text in alphabet $\Sigma$ and you compress one input symbol from the alphabet at a time.

- Could you adapt the algorithm so that it utilizes three output symbols $\{0,1,2\}$?
- What about $k$ target symbols ($k < $ alphabet size)?
-  What would happen if you used $\Sigma$ as the output alphabet for Huffman encoding? Would the text remain the same? Would it have the same length?
- What changes if the input is words (i.e. sequences from the $\Sigma$ alphabet with one symbol representing the word boundary)?
- What changes if you are allowed to use phrases (i.e. ignore the word boundary) if it helps compression?

Answer these questions in 1-2 sentences max.


Unit 6

Assessing model accuracy

(Notebook: regression_model_comparison.ipynb)

IST 718 – Big Data Analytics

Daniel E. Acuna

http://acuna.io

# From previous unit
- We took a **statistical approach to learning** which acknowledges our uncertainty and noise in the data science process.

- We defined a **model** of the data.

- We estimated the parameters using **training data**.

- We used the model to **predict** and **interpret** the results.

- We could use **supervised** or **unsupervised** learning to find relationships between variables.

- If variables are not quantitative, we used classification models.

# In this unit
- Generalization performance
- Estimating generalization performance: cross-validation
- Bias-variance decomposition
- Performance of classifiers: confusion matrix, ROC curve, AUC
- Bayes rule

# Generalization performance
- Generalization is the performance of a learning method on independent **test data**.

- Generalization performance guides the choice of learning method or model.
- **Why don't we teach the *best method*?**
  - *Because there is no free lunch in statistics*: David Wolpert, "The Lack of A Priori Distinctions Between Learning Algorithms", 1996.

    No one method dominates all others over all possible data sets.

# Generalization performance (2)
- The no free lunch theorem implies that we need to:

  1. Learn about the **particular dataset** we are working on (data science!)

  2. **Select the best method** using generalization performance (data science!)

# Measuring generalization performance: theory
- We need to define a loss function:

$$l(Y,\hat{f}(X))$$

- For example, the Squared Error for regression:

$$l(Y,\hat{f}(X)) = (Y - \hat{f}(X))^2$$

(which is the same as the negative likelihood with Gaussian noise.)

- Or zero-one loss for classification:

$$l(Y,\hat{f}(X)) = \text{I}(Y \neq \hat{f}(X))$$

where $\text{I}(a,b)$ is 1 if $a = b$, and 0 otherwise.

# Measuring generalization performance: test error and expected prediction error
- Test error is the prediction error over an *independent* test sample:

$$Err_T = E[l(Y,\hat{f}(X)) \mid T]$$

where both $Y$ and $X$ are randomly sampled with a fixed training set $T$.

- A related quantity is the expected prediction error:

$$Err = E[l(Y,\hat{f}(X))] = E[Err_T]$$

where everything is random including the training dataset.

- Most methods effectively estimate $Err$ instead of $Err_T$.

# Measuring generalization performance: model comparison

- Typically, we must compare several models
- We split the data into **three datasets** and compute the following over a **testing dataset**
$$Err_{V,T}=E[l(Y,\hat{f}(X))\mid V,T ]$$
where $V$ is a **validation dataset**, and $T$ is a **training dataset**.
- We further restrict the previous quantity to the best model on **validation** performance, and therefore we end up estimating
$$Err = E[E_T[Err_{V^*,T}]]$$

# Estimating test error in practice: cross validation

Often, models have differing degrees of complexity controlled by a parameter $\alpha$ (e.g., $\;\hat{f}_\alpha(X)$)

Training split is used to **fit** one model.

Validation split is used to select **complexity**.

Test split is used to estimate **expected test error**.

# Estimating test error in practice: cross validation (2)
- The previous approach is known as **training, validation, and testing split**
- If no alternative models are compared, there are only two splits and the method is known as training and testing
- Typical data splits are 60%-30%-10% for training, validation, and testing
- Or 80%-20% for training and testing splits

# Estimating the expected test error: $k$-fold cross validation

The problem with the previous procedure is that we "throw away" the validation and test splits during training.

$k$-fold cross validation (partially) fixes this by running cross validation multiple times.

# Estimating the expected test error: training data splits
- The **test split** should **only** be used at the end of the data science.
- (Demo) We will compare two models in diabetes dataset: linear regression using BMI (M1) and linear regression using BMI and age (M2).

  1. Model fit:
  M1 MSE (training) = 3723, **M2 MSE  (training) = 3680**

  2. Model selection:
  **M1 MSE (validation) = 4582**, M2 MSE (validation) = 4618

  3. Model assessment:
  **M1 MSE (test) = 3925**

# Estimating the expected test error: variable selection
- This procedure can be generalized to answer the question:
  - Which variables should be included in the model?

- One of the simplest such procedures is the null-to-full model variable selection:

For $k$ variables from 0 to $p$:

Fit $p – k$ models which add one variable to the current model.

If none of the models has lower validation error than current model, then break.

Set the current model to the one with lowest validation error.

Return current model.

- This procedure selects age, bmi, map, tc, ltg, and glu as the most important variables with estimated test error of **3324**.

# More on expected test error
- The loss function used to cross validation **does not** necessarily match the loss function used during model fitting.
- For example, you can fit models with gradient descent to minimize MSE (because it is easy and fast) but you might choose models based on the Mean Absolute Deviation (MAD)
$$\text{MAD} = \frac{1}{n} \sum_{i=1}^n | \hat{y}_i - y |$$

# The Bias-Variance decomposition: Math
- Mathematically:

$$\begin{align}
Err(x_0) &= E[(Y-\hat{f}(x_0))^2\mid x_0] \\
Err(x_0) &= \sigma_{\epsilon}^2 + (E[\hat{f}(x_0)]-f(x_0))^2 + E[\hat{f}(x_0)-E[\hat{f}(x_0)]]^2 \\
Err(x_0) &= \text{Irreducible error} + \text{Bias}^2 + \text{Variance}
\end{align}$$

# The Bias-Variance decomposition of test error
- In general, more complex models have low bias and high variance.

- Vice versa, simple models have high bias and low variance.

- This is a **fundamental tradeoff**.

- **High variance** means that the estimation has high "error bars."

- **High bias** means that the estimation will not change much even if more data is seen.

# The Bias-Variance decomposition: Nearest neighbor regression
- Take the $k$ closest points and compute the average $y$ of those points
$$\hat{f}(x_0) = \frac{1}{k}\sum_{i=1}^k f(x_{\text{nn}_i(x_0)})$$
where $\text{nn}_i(x)$ is the index of $i$-th closest neighbor to $x$
- The Bias-variance decomposition then becomes
$$Err(x_0) = \sigma_\epsilon^2 + [f(x_0) - \frac{1}{k}\sum_{i=1}^k f(x_{\text{nn}_i(x_0)})]^2 + \frac{\sigma_\epsilon^2}{k}$$
where $\frac{\sigma_\epsilon^2}{k}$ is the variance of $\hat{f}$
- Bias increases with $k$ and variance decreases with $k$

# The Bias-Variance decomposition: Linear regression
- The argument is more complex: $\hat{f}(x_0) = x_0^T b$ with vector $b$ having $p$ components
- On average
$$Err(x_0) \approx \sigma_\epsilon^2 + \text{Bias}^2 + p \sigma_\epsilon^2$$
- Variance is proportional to the number of parameters and therefore Bias decreases with number of parameters

# The Bias-Variance decomposition: in practice
- It is many times impossible to know the irreducible error, bias, and variance decomposition
- There are several heuristics for trying to understand whether we need to increase the bias or the variance, or whether we have simply hit the irreducible error
- We will want to **search over many models to find the one which minimizes bias and variance**

# The Bias-Variance decomposition (2)
An example with the diabetes dataset:
- We add a polynomial expansion to the variables:
  - bmi, age, bmi*age, bmi$^2$, age$^2$, etc.

- We will have a new set of features of size:

$$p_\text{new} = 2p + p(p-1)/2$$

- We run the following procedure:
  Repeat many times:
  1. Randomly split the data into training and testing
  2. For $k=1$ to $p_\text{new}$
    1. Fit model with $k$ randomly selected features and predict.
    2. Estimate training and testing MSEs.

# The Bias-Variance decomposition (3)

# The Bias-Variance decomposition: the learning curve

Simple model doesn’t learn much after 50 examples.

Complex model keeps learning.

Simple model has better performance with small datasets.

Complex model has better performance with big datasets.

# Measuring generalization performance: classification
- A typical method for measuring classification performance is *accuracy*:

    $$\text{Accuracy} = \frac{1}{n} \sum_{i=1}^{n} \text{I}(y_i,\hat{f}(x_i))$$
where $\text{I}(a, b)$ is 1 if $a = b$, and 0 otherwise.

- What might be the problem with using accuracy? (hint: What is the accuracy of an algorithm that predicts that every email received is not a spam?)

# Measuring generalization performance: confusion matrix
- **Activity**: Predict spam. Given one normal email (True Normal: TN) and one spam email (True Spam: TS), Gmail can classify as spam (Predict Spam: PS) or not spam (Predict Normal: PN). Where would you put all these cases in a confusion matrix?

# Measuring generalization performance: confusion matrix (2)
- **Activity**: Predict spam. Given one normal email (True Normal: TN) and one spam email (True Spam: TS), Gmail can classify as spam (Predict Spam: PS) or not spam (Predict Normal: PN). Where would you put all these cases in a confusion matrix?

# Measuring generalization performance: confusion matrix (3)
- This matrix is called a **confusion matrix**: summary of true vs predicted cases

# Measuring generalization performance: Common statistics from confusion matrix
- **Prevalence**: (TP+FN) / everything
- **Precision**: TP / (TP + FP)
- **Sensitivity**, **Recall**, or **True positive rate**: TP / true condition positive
- **Specificity**: TN / true condition negative
- **F1**: $\frac{2*precision*recall}{precision + recall}$

# Measuring generalization performance: ROC curve
- In general, no single algorithm has the best sensitivity and specificity simultaneously.
- Some algorithms offer a range of sensitivity/specificity points.  ROC curve displays this

# Measuring generalization performance: ROC curve (2)
- We will classify as spam if classifier thinks with more than 50% probability.

# Measuring generalization performance: ROC curve (3)
- **Less stringent, we will classify as spam if P(spam) > 0.1**

# Measuring generalization performance: ROC curve (4)
- We can put the two thresholds $\theta = 0.5$ and $\theta = 0.1$ in a plot.

# Measuring generalization performance: ROC curve  (5)
- We can do this for every threshold value and plot the result as a curve.

**Activity**: can you guess the curve of a random classifier?

# Measuring generalization performance: ROC curve (6)
- We can do this for every threshold value and plot the result as follows:

A measure that combines all thresholds is the **Area Under the ROC Curve (AUC)**

# Area under the ROC curve (AUC)

- It is hard to interpret but:
    - It can be thought as how good is the model to rank the probabilities with the real labels
- The **AUC** is 1/2 if the classifier is *random* or *constant*
- **Activity: show that the AUC is in fact 1/2 for a random predictor**

# AUC for random prediction

- AP, AN, PP, PN mean actual positive, actual negative, predicted positive, and predictived negative, respectively
- There are $n$ cases in total

| .  | PP|PN |
|---|---|---|
| AP| TP|FN |
| AN| FP|TN |

- Cases are gonna randomly fall on PP and PN.
- For a given threshold $\theta$, FP = $(1-\theta)$AN and TN=$\theta$AN, and similarly TP=$(1-\theta)$AP and FN=$\theta$AP
- FPR = $(1-\theta)$AN/($(1-\theta)$AN + $\theta$AN) = $1-\theta$
- Therefore, AUC = $\frac{1}{2}$

# AUC for constant prediction
- As we move the threshold there will be a change between all points being in PP to PN, or viceverse.
- There two points will have FPR=TPR=0 and FPR=TPR=1 respectively
- By interpolation, AUC = $\frac{1}{2}$

# The Bayesian classifier
- Many values from confusion matrix are misleading because we are not using *prevalence rates*.

- Maximum likelihood estimators estimate the most likely classification of the data given a hypothesis (actual class) without consideration of the *prevalance rate*

- The optimal decision however is achieved by Bayes' theorem

$$p(\text{hypothesis} \mid \text{data}) = \frac{p(\text{data} \mid \text{hypothesis})\;p(\text{hypothesis})}{p(\text{data})}$$

# The Bayesian classifier (2)
- Sometimes classification performance in terms of metrics that do not consider prevalance.
- For example, a classifier for cancer (C) given positive results (R) or might be reported with:
  - Specificity: 95%
  - Sensitivity: 60%
- But cancer prevalance is low (0.2%)
$$p(\text{C}\mid \text{R}) = \frac{p(\text{R}\mid\text{C})p(\text{C})}{p(\text{R})}$$
with
$$p(\text{R}) = p(\text{R}\mid\text{C})p(\text{C}) + p(\text{R}\mid\neg\text{C})p(\neg\text{C})$$

# Take home message
- Always keep the testing dataset in vault until the end of your analysis.

- Use model selection to choose from models of different complexity.

- Stick to a loss function throughout your analysis.

- For classification problems, think carefully about the requirements of your problem.

- Be careful about the prevalence values.


```python
from IPython.display import Image
Image('../../../python_for_probability_statistics_and_machine_learning.jpg')
```

[Python for Probability, Statistics, and Machine Learning](https://www.springer.com/fr/book/9783319307152)

# Moment Generating Functions

Generating moments usually involves integrals that are extremely
difficult to compute. Moment generating functions make this much, much
easier. The moment generating function is defined as,

$$
M(t) = \mathbb{E}(\exp(t X))
$$

 The first moment is the mean, which we can easily compute from
$M(t)$ as,

$$
\begin{align*}
\frac{dM(t)}{dt} &= \frac{d}{dt}\mathbb{E}(\exp(t X)) = \mathbb{E}\frac{d}{dt}(\exp(t X))\\\
                 &= \mathbb{E}(X \exp(t X))  \\\
\end{align*}
$$

 Now, we have to set $t=0$ and we have the mean,

$$
M^{(1)}(0) = \mathbb{E}(X)
$$

 continuing this derivative process again, we obtain the second moment as,

$$
\begin{align*}
M^{(2)}(t) &= \mathbb{E}(X^2\exp(t X)) \\\
M^{(2)}(0) &= \mathbb{E}(X^2)
\end{align*}
$$

 With this in hand, we can easily compute the variance as,

$$
\mathbb{V}(X) = \mathbb{E}(X^2) -\mathbb{E}(X)^2=M^{(2)}(0)-M^{(1)}(0)^2
$$

**Example.** Returning to our favorite binomial distribution, let's compute
some moments using Sympy.

```python
import sympy as S
from sympy import stats
p,t = S.symbols('p t',positive=True)
x=stats.Binomial('x',10,p)
mgf = stats.E(S.exp(t*x))
```

 Now, let's compute the first moment (aka, mean) using
the usual integration method and using moment generating functions,

```python
print S.simplify(stats.E(x))
print S.simplify(S.diff(mgf,t).subs(t,0))
```

    10*p
    10*p

 Otherwise, we can compute this directly as follows,

```python
print S.simplify(stats.moment(x,1)) # mean
print S.simplify(stats.moment(x,2)) # 2nd moment
```

    10*p
    10*p*(9*p + 1)

 In general, the moment generating function for the binomial
distribution is the following,

$$
M_X(t) = \left(p\left(e^t-1\right)+1\right) ^n
$$

A key aspect of moment generating functions is that they are unique identifiers
of probability distributions.  By the uniqueness theorem, given two random
variables $X$ and $Y$, if their respective moment generating functions are
equal, then the corresponding probability distribution functions are equal.

**Example.** Let's use the uniqueness theorem to consider the following
problem. Suppose we know that the probability distribution of $X$ given $U=p$
is binomial with parameters $n$ and $p$. For example, suppose $X$ represents the
number of heads in $n$ coin flips, given the probability of heads is $p$. We
want to find the unconditional distribution of $X$. Writing out the
moment generating function as the following,

$$
\mathbb{E}(e^{t X}\vert U=p) = (p e^t + 1-p)^n
$$

 Because $U$ is uniform over the unit interval, we can
integrate this part out

$$
\begin{align*}
\mathbb{E}(e^{t X}) &=\int_0^1 (p e^t + 1-p)^n dp \\\
                    &= \frac{1}{n+1} \frac{e^{t(n+1)-1}}{e^t-1} \\\
                    &= \frac{1}{n+1} (1+e^t+e^{2t}+e^{3t}+\ldots+e^{n t}) \\\
\end{align*}
$$

 Thus, the moment generating function of $X$ corresponds to that of a
random variable that is equally likely to be any of the values $0,1,\ldots,n$.
This is another way of saying that the distribution of $X$ is discrete uniform
over $\lbrace 0,1,\ldots,n \rbrace$. Concretely, suppose we have a box of coins
whose individual probability of heads is unknown and  that we  dump the box on
the floor, spilling all of the coins. If we then count the number of coins facing
heads-up, that distribution is uniform.

Moment generating functions are useful for deriving distributions of
sums of independent random variables. Suppose $X_1$ and $X_2$ are independent
and $Y=X_1+X_2$. Then, the moment generating function of $Y$ follows
from the properties of the expectation,

$$
\begin{align*}
M_Y(t) &= \mathbb{E}(e^{t Y}) =  \mathbb{E}(e^{t X_1 + t X_2}) \\\
       &= \mathbb{E}(e^{t X_1} e^{ t X_2 }) =\mathbb{E}(e^{t X_1})\mathbb{E}(e^{t X_2}) \\\
       &= M_{X_1}(t)M_{X_2}(t)
\end{align*}
$$

**Example.** Suppose we have two normally distributed random variables,
$X_1\sim \mathcal{N}(\mu_1,\sigma_1)$ and $ X_2\sim \mathcal{N}(\mu_2,\sigma_2)$.
We can save some tedium by exploring this in Sympy,

```python
S.var('x:2',real=True)
S.var('mu:2',real=True)
S.var('sigma:2',positive=True)
S.var('t',positive=True)
x0=stats.Normal(x0,mu0,sigma0)
x1=stats.Normal(x1,mu1,sigma1)
```

**Programming Tip.**

The `S.var` function defines the variable and injects it into the global
namespace. This is sheer laziness. It is more expressive to define variables
explicitly as in `x = S.symbols('x')`. Also notice that we used the Greek names
for the `mu` and `sigma` variables.  This will come in handy later when we want
to render the equations in the Jupyter/IPython notebook which understands
how to typeset these symbols in \LaTeX{}. The `var('x:2')` creates two
symbols, `x0` and `x1`. Using the colon this way makes it easy to generate
array-like sequences of symbols.

 In the next block we compute the moment generating functions

```python
mgf0=S.simplify(stats.E(S.exp(t*x0)))
mgf1=S.simplify(stats.E(S.exp(t*x1)))
mgfY=S.simplify(mgf0*mgf1)
```

 The moment generating functions an individual normally distributed
random variable is the following,

$$
e^{\mu_{0} t + \frac{\sigma_{0}^{2} t^{2}}{2}}
$$

 Note the coefficients of $t$. To show that $Y$ is normally
distributed, we want to match the moment generating function of $Y$ to this
format. The following is the form of the moment generating function of $Y$,

$$
M_Y(t)=e^{\frac{t}{2} \left(2 \mu_{0} + 2 \mu_{1} + \sigma_{0}^{2} t + \sigma_{1}^{2} t\right)}
$$

 We can extract the exponent using Sympy and collect on the $t$
variable using the following code,

```python
S.collect(S.expand(S.log(mgfY)),t)
```

    t**2*(sigma0**2/2 + sigma1**2/2) + t*(mu0 + mu1)

 Thus, by the uniqueness theorem, $Y$ is normally distributed with
$\mu_Y=\mu_0+\mu_1$ and $\sigma_Y^2=\sigma_0^2+\sigma_1^2$.

**Programming Tip.**

When using the Jupyter/IPython notebook, you can do `S.init_printing` to get
the mathematical typesetting to work in the browser. Otherwise, if you want to
keep the raw expression and to selectively render to \LaTeX{}, then you can
`from IPython.display import Math`, and then use `Math(S.latex(expr))` to see
the typeset version of the expression.


```python
# Erasmus+ ICCT project (2018-1-SI01-KA203-047081)

# Toggle cell visibility

from IPython.display import HTML
tag = HTML('''
Toggle cell visibility <a href="javascript:code_toggle()">here</a>.''')
display(tag)

# Hide the code completely

# from IPython.display import HTML
# tag = HTML('''<style>
# div.input {
#     display:none;
# }
# </style>''')
# display(tag)

```

Toggle cell visibility here.

## State feedback control

This example shows the effect of full state feedback.

Given the linear time invariant system:

\begin{cases}
\dot{\textbf{x}}=A\textbf{x}+B\textbf{u} \\
\textbf{y}=C\textbf{x},
\end{cases}

and the control law:

$\textbf{u}=-K\textbf{x}+\textbf{v},$

this example shows the free and forced responses of the close loop system:

$$
\dot{\textbf{x}}=A\textbf{x}-BK\textbf{x}+B\textbf{v} = (A-BK)\textbf{x}+B\textbf{v}.
$$

### How to use this notebook?
Try to change the values of $K$ or directly set the eigenvalues of $(A-BK)$ and get the corresponding controller gains:
- Create an unstable system and make it stable with full state feedback.
- Create a system with a slow response and make it faster with full state feedback.
- Create a system that is not fully controllable try to change all its eigenvalues in closed loop. Is it possible to achieve that?
- Create a system that is not fully controllable and unstable and try to make it stable with full state feedback. Can you tell in which cases it is possible to stabilize it with closed-loop control?

```python
%matplotlib inline
import control as control
import numpy
import sympy as sym
from IPython.display import display, Markdown
import ipywidgets as widgets
import matplotlib.pyplot as plt

#print a matrix latex-like
def bmatrix(a):
     """Returns a LaTeX bmatrix - by Damir Arbula (ICCT project)

     :a: numpy array
     :returns: LaTeX bmatrix as a string
     """
     if len(a.shape) > 2:
         raise ValueError('bmatrix can at most display two dimensions')
     lines = str(a).replace('[', '').replace(']', '').splitlines()
     rv = [r'\begin{bmatrix}']
     rv += ['  ' + ' & '.join(l.split()) + r'\\' for l in lines]
     rv +=  [r'\end{bmatrix}']
     return '\n'.join(rv)

# Display formatted matrix:
def vmatrix(a):
    if len(a.shape) > 2:
         raise ValueError('bmatrix can at most display two dimensions')
    lines = str(a).replace('[', '').replace(']', '').splitlines()
    rv = [r'\begin{vmatrix}']
    rv += ['  ' + ' & '.join(l.split()) + r'\\' for l in lines]
    rv +=  [r'\end{vmatrix}']
    return '\n'.join(rv)

#matrixWidget is a matrix looking widget built with a VBox of HBox(es) that returns a numPy array as value !
class matrixWidget(widgets.VBox):
    def updateM(self,change):
        for irow in range(0,self.n):
            for icol in range(0,self.m):
                self.M_[irow,icol] = self.children[irow].children[icol].value
                #print(self.M_[irow,icol])
        self.value = self.M_

    def dummychangecallback(self,change):
        pass

    def __init__(self,n,m):
        self.n = n
        self.m = m
        self.M_ = numpy.matrix(numpy.zeros((self.n,self.m)))
        self.value = self.M_
        widgets.VBox.__init__(self,
                             children = [
                                 widgets.HBox(children =
                                              [widgets.FloatText(value=0.0, layout=widgets.Layout(width='90px')) for i in range(m)]
                                             )
                                 for j in range(n)
                             ])

        #fill in widgets and tell interact to call updateM each time a children changes value
        for irow in range(0,self.n):
            for icol in range(0,self.m):
                self.children[irow].children[icol].value = self.M_[irow,icol]
                self.children[irow].children[icol].observe(self.updateM, names='value')
        #value = Unicode('example@example.com', help="The email value.").tag(sync=True)
        self.observe(self.updateM, names='value', type= 'All')

    def setM(self, newM):
        #disable callbacks, change values, and reenable
        self.unobserve(self.updateM, names='value', type= 'All')
        for irow in range(0,self.n):
            for icol in range(0,self.m):
                self.children[irow].children[icol].unobserve(self.updateM, names='value')
        self.M_ = newM
        self.value = self.M_
        for irow in range(0,self.n):
            for icol in range(0,self.m):
                self.children[irow].children[icol].value = self.M_[irow,icol]
        for irow in range(0,self.n):
            for icol in range(0,self.m):
                self.children[irow].children[icol].observe(self.updateM, names='value')
        self.observe(self.updateM, names='value', type= 'All')

                #self.children[irow].children[icol].observe(self.updateM, names='value')

#overlaod class for state space systems that DO NOT remove "useless" states (what "professor" of automatic control would do this?)
class sss(control.StateSpace):
    def __init__(self,*args):
        #call base class init constructor
        control.StateSpace.__init__(self,*args)
    #disable function below in base class
    def _remove_useless_states(self):
        pass
```

```python
# Preparatory cell

A = numpy.matrix('0 1 0; 0 0 1; 0 2 -3')
B = numpy.matrix('0; 0; 1')
C = numpy.matrix('1 0 0; 0 1 0; 0 0 1')
X0 = numpy.matrix('2; 2; 2')
K = numpy.matrix([8,14,3])
sol1 = numpy.linalg.eig(A)

Aw = matrixWidget(3,3)
Aw.setM(A)
Bw = matrixWidget(3,1)
Bw.setM(B)
Cw = matrixWidget(3,3)
Cw.setM(C)
X0w = matrixWidget(3,1)
X0w.setM(X0)
Kw = matrixWidget(1,3)
Kw.setM(K)

eig1c = matrixWidget(1,1)
eig2c = matrixWidget(2,1)
eig3c = matrixWidget(1,1)
eig1c.setM(numpy.matrix([-2]))
eig2c.setM(numpy.matrix([[-2],[0]]))
eig3c.setM(numpy.matrix([-2]))
```

```python
# Misc

#create dummy widget
DW = widgets.FloatText(layout=widgets.Layout(width='0px', height='0px'))

#create button widget
START = widgets.Button(
    description='Test',
    disabled=False,
    button_style='', # 'success', 'info', 'warning', 'danger' or ''
    tooltip='Test',
    icon='check'
)

def on_start_button_clicked(b):
    #This is a workaround to have intreactive_output call the callback:
    #   force the value of the dummy widget to change
    if DW.value> 0 :
        DW.value = -1
    else:
        DW.value = 1
    pass
START.on_click(on_start_button_clicked)

# Define type of method
selm = widgets.Dropdown(
    options= ['Set K', 'Set the eigenvalues'],
    value= 'Set K',
    description='',
    disabled=False
)

# Define the number of complex eigenvalues for the observer
selc = widgets.Dropdown(
    options= ['0 complex eigenvalues', '2 complex eigenvalues'],
    value= '0 complex eigenvalues',
    description='Eigenvalues:',
    disabled=False
)

#define type of ipout
selu = widgets.Dropdown(
    options=['impulse', 'step', 'sinusoid', 'square wave'],
    value='impulse',
    description='Type of input:',
    disabled=False
)
# Define the values of the input
u = widgets.FloatSlider(
    value=1,
    min=0,
    max=20.0,
    step=0.1,
    description='input u:',
    disabled=False,
    continuous_update=False,
    orientation='horizontal',
    readout=True,
    readout_format='.1f',
)
period = widgets.FloatSlider(
    value=0.5,
    min=0.05,
    max=1,
    step=0.05,
    description='Period: ',
    disabled=False,
    continuous_update=False,
    orientation='horizontal',
    readout=True,
    readout_format='.2f',
)
```

```python
# Support functions

def eigen_choice(selc):
    if selc == '0 complex eigenvalues':
        eig1c.children[0].children[0].disabled = False
        eig2c.children[1].children[0].disabled = True
        eigc = 0
    if selc == '2 complex eigenvalues':
        eig1c.children[0].children[0].disabled = True
        eig2c.children[1].children[0].disabled = False
        eigc = 2
    return eigc

def method_choice(selm):
    if selm == 'Set K':
        method = 1
        selc.disabled = True
    if selm == 'Set the eigenvalues':
        method = 2
        selc.disabled = False
    return method
```

```python
def main_callback(Aw, Bw, X0w, K, eig1c, eig2c, eig3c, u, period, selm, selc, selu, DW):
    A, B = Aw, Bw
    sols = numpy.linalg.eig(A)
    eigc = eigen_choice(selc)
    method = method_choice(selm)

    if method == 1:
        sol = numpy.linalg.eig(A-B*K)
    if method == 2:
        if eigc == 0:
            K = control.acker(A, B, [eig1c[0,0], eig2c[0,0], eig3c[0,0]])
            Kw.setM(K)
        if eigc == 2:
            K = control.acker(A, B, [eig1c[0,0],
                                      numpy.complex(eig2c[0,0],eig2c[1,0]),
                                      numpy.complex(eig2c[0,0],-eig2c[1,0])])
            Kw.setM(K)
        sol = numpy.linalg.eig(A-B*K)
    print('The system\'s eigenvalues are:',round(sols[0][0],4),',',round(sols[0][1],4),'and',round(sols[0][2],4))
    print('The controlled system\'s eigenvalues are:',round(sol[0][0],4),',',round(sol[0][1],4),'and',round(sol[0][2],4))

    sys = sss(A,B,C,sym.zeros(3,1))
    sysc = sss(A-B*K, B, numpy.eye(3), numpy.zeros(3).reshape((3,1)))
    T = numpy.linspace(0, 6, 1000)

    if selu == 'impulse': #selu
        U = [0 for t in range(0,len(T))]
        U[0] = u
        T, yout, xout = control.forced_response(sys,T,U,X0w)
        T, youtc, xoutc = control.forced_response(sysc,T,U,X0w)
    if selu == 'step':
        U = [u for t in range(0,len(T))]
        T, yout, xout = control.forced_response(sys,T,U,X0w)
        T, youtc, xoutc = control.forced_response(sysc,T,U,X0w)
    if selu == 'sinusoid':
        U = u*numpy.sin(2*numpy.pi/period*T)
        T, yout, xout = control.forced_response(sys,T,U,X0w)
        T, youtc, xoutc = control.forced_response(sysc,T,U,X0w)
    if selu == 'square wave':
        U = u*numpy.sign(numpy.sin(2*numpy.pi/period*T))
        T, yout, xout = control.forced_response(sys,T,U,X0w)
        T, youtc, xoutc = control.forced_response(sysc,T,U,X0w)

    fig = plt.figure(num='Simulation', figsize=(16,10))

    fig.add_subplot(311)
    plt.ylabel('$X_1$ vs $X_{1f}$')
    plt.plot(T,xout[0])
    plt.plot(T,xoutc[0])
    plt.xlabel('time [s]')
    plt.legend(['Open Loop','State Feedback'])
    plt.axvline(x=0,color='black',linewidth=0.8)
    plt.axhline(y=0,color='black',linewidth=0.8)
    plt.grid()

    fig.add_subplot(312)
    plt.ylabel('$X_2$ vs $X_{2f}$')
    plt.plot(T,xout[1])
    plt.plot(T,xoutc[1])
    plt.xlabel('time [s]')
    plt.legend(['Open Loop','State Feedback'])
    plt.axvline(x=0,color='black',linewidth=0.8)
    plt.axhline(y=0,color='black',linewidth=0.8)
    plt.grid()

    fig.add_subplot(313)
    plt.ylabel('$X_3$ vs $X_{3f}$')
    plt.plot(T,xout[2])
    plt.plot(T,xoutc[2])
    plt.xlabel('time [s]')
    plt.legend(['Open Loop','State Feedback'])
    plt.axvline(x=0,color='black',linewidth=0.8)
    plt.axhline(y=0,color='black',linewidth=0.8)
    plt.grid()

alltogether = widgets.VBox([widgets.HBox([selm,
                                          selc,
                                          selu]),
                            widgets.Label(' ',border=3),
                            widgets.HBox([widgets.Label('K:',border=3), Kw,
                                          widgets.Label(' ',border=3),
                                          widgets.Label(' ',border=3),
                                          widgets.Label('Eigenvalues:',border=3),
                                          eig1c,
                                          eig2c,
                                          eig3c,
                                          widgets.Label(' ',border=3),
                                          widgets.Label(' ',border=3),
                                          widgets.Label('X0:',border=3), X0w]),
                            widgets.Label(' ',border=3),
                            widgets.HBox([u,
                                          period,
                                          START]),
                            widgets.Label(' ',border=3),
                            widgets.HBox([widgets.Label('Dynamics matrix A:',border=3),
                                          Aw,
                                          widgets.Label('Input matrix B:',border=3),
                                          Bw])])
out = widgets.interactive_output(main_callback, {'Aw':Aw, 'Bw':Bw, 'X0w':X0w, 'K':Kw, 'eig1c':eig1c, 'eig2c':eig2c, 'eig3c':eig3c,
                                                 'u':u, 'period':period, 'selm':selm, 'selc':selc, 'selu':selu, 'DW':DW})
out.layout.height = '680px'
display(out, alltogether)
```

    Output(layout=Layout(height='680px'))

    VBox(children=(HBox(children=(Dropdown(options=('Set K', 'Set the eigenvalues'), value='Set K'), Dropdown(desc…

```python

```


```python
import numpy as np

# Importing standard Qiskit libraries
from qiskit import QuantumCircuit, transpile, Aer, IBMQ
from qiskit.tools.jupyter import *
from qiskit.visualization import *
from ibm_quantum_widgets import *
from qiskit.providers.aer import QasmSimulator

# Loading your IBM Quantum account(s)
provider = IBMQ.load_account()
```

```python
IBMQ.providers()
```

    [<AccountProvider for IBMQ(hub='ibm-q', group='open', project='main')>]

```python
provider=IBMQ.get_provider(hub='ibm-q')
```

```python
provider
```

    <AccountProvider for IBMQ(hub='ibm-q', group='open', project='main')>

```python
type(provider)
```

    qiskit.providers.ibmq.accountprovider.AccountProvider

```python
print(type(provider))
```

    <class 'qiskit.providers.ibmq.accountprovider.AccountProvider'>

```python
provider=IBMQ.get_provider(group='open')
```

```python
provider
```

    <AccountProvider for IBMQ(hub='ibm-q', group='open', project='main')>

```python
type(provider)
```

    qiskit.providers.ibmq.accountprovider.AccountProvider

```python
print(type(provider))
```

    <class 'qiskit.providers.ibmq.accountprovider.AccountProvider'>

```python
provider.backends()
```

    [<IBMQSimulator('ibmq_qasm_simulator') from IBMQ(hub='ibm-q', group='open', project='main')>,
     <IBMQBackend('ibmq_armonk') from IBMQ(hub='ibm-q', group='open', project='main')>,
     <IBMQBackend('ibmq_santiago') from IBMQ(hub='ibm-q', group='open', project='main')>,
     <IBMQBackend('ibmq_bogota') from IBMQ(hub='ibm-q', group='open', project='main')>,
     <IBMQBackend('ibmq_lima') from IBMQ(hub='ibm-q', group='open', project='main')>,
     <IBMQBackend('ibmq_belem') from IBMQ(hub='ibm-q', group='open', project='main')>,
     <IBMQBackend('ibmq_quito') from IBMQ(hub='ibm-q', group='open', project='main')>,
     <IBMQSimulator('simulator_statevector') from IBMQ(hub='ibm-q', group='open', project='main')>,
     <IBMQSimulator('simulator_mps') from IBMQ(hub='ibm-q', group='open', project='main')>,
     <IBMQSimulator('simulator_extended_stabilizer') from IBMQ(hub='ibm-q', group='open', project='main')>,
     <IBMQSimulator('simulator_stabilizer') from IBMQ(hub='ibm-q', group='open', project='main')>,
     <IBMQBackend('ibmq_manila') from IBMQ(hub='ibm-q', group='open', project='main')>]

```python
provider.backends(simulator=False, operational=True)
```

    [<IBMQBackend('ibmq_armonk') from IBMQ(hub='ibm-q', group='open', project='main')>,
     <IBMQBackend('ibmq_bogota') from IBMQ(hub='ibm-q', group='open', project='main')>,
     <IBMQBackend('ibmq_lima') from IBMQ(hub='ibm-q', group='open', project='main')>,
     <IBMQBackend('ibmq_belem') from IBMQ(hub='ibm-q', group='open', project='main')>,
     <IBMQBackend('ibmq_quito') from IBMQ(hub='ibm-q', group='open', project='main')>,
     <IBMQBackend('ibmq_manila') from IBMQ(hub='ibm-q', group='open', project='main')>]

```python
provider.backends(simulator=True, operational=True)
```

    [<IBMQSimulator('ibmq_qasm_simulator') from IBMQ(hub='ibm-q', group='open', project='main')>,
     <IBMQSimulator('simulator_statevector') from IBMQ(hub='ibm-q', group='open', project='main')>,
     <IBMQSimulator('simulator_mps') from IBMQ(hub='ibm-q', group='open', project='main')>,
     <IBMQSimulator('simulator_extended_stabilizer') from IBMQ(hub='ibm-q', group='open', project='main')>,
     <IBMQSimulator('simulator_stabilizer') from IBMQ(hub='ibm-q', group='open', project='main')>]

```python
provider.backends(simulator=False, operational=False)
```

    [<IBMQBackend('ibmq_santiago') from IBMQ(hub='ibm-q', group='open', project='main')>]

```python
provider.backends(filters=lambda x:x.configuration().n_qubits >=5 and not x.configuration().simulator and x.status().operational==True)
```

    [<IBMQBackend('ibmq_bogota') from IBMQ(hub='ibm-q', group='open', project='main')>,
     <IBMQBackend('ibmq_lima') from IBMQ(hub='ibm-q', group='open', project='main')>,
     <IBMQBackend('ibmq_belem') from IBMQ(hub='ibm-q', group='open', project='main')>,
     <IBMQBackend('ibmq_quito') from IBMQ(hub='ibm-q', group='open', project='main')>,
     <IBMQBackend('ibmq_manila') from IBMQ(hub='ibm-q', group='open', project='main')>]

```python
backend = provider.backends()
```

```python
backend
```

    [<IBMQSimulator('ibmq_qasm_simulator') from IBMQ(hub='ibm-q', group='open', project='main')>,
     <IBMQBackend('ibmq_armonk') from IBMQ(hub='ibm-q', group='open', project='main')>,
     <IBMQBackend('ibmq_santiago') from IBMQ(hub='ibm-q', group='open', project='main')>,
     <IBMQBackend('ibmq_bogota') from IBMQ(hub='ibm-q', group='open', project='main')>,
     <IBMQBackend('ibmq_lima') from IBMQ(hub='ibm-q', group='open', project='main')>,
     <IBMQBackend('ibmq_belem') from IBMQ(hub='ibm-q', group='open', project='main')>,
     <IBMQBackend('ibmq_quito') from IBMQ(hub='ibm-q', group='open', project='main')>,
     <IBMQSimulator('simulator_statevector') from IBMQ(hub='ibm-q', group='open', project='main')>,
     <IBMQSimulator('simulator_mps') from IBMQ(hub='ibm-q', group='open', project='main')>,
     <IBMQSimulator('simulator_extended_stabilizer') from IBMQ(hub='ibm-q', group='open', project='main')>,
     <IBMQSimulator('simulator_stabilizer') from IBMQ(hub='ibm-q', group='open', project='main')>,
     <IBMQBackend('ibmq_manila') from IBMQ(hub='ibm-q', group='open', project='main')>]

```python
backend.name()
```

    Traceback (most recent call last):
      Input In [38] in <module>
        backend.name()
    AttributeError: 'list' object has no attribute 'name'

    Use %tb to get the full traceback.

<button class='button iqx-button'>Search for solution online</button>

```python
backend.version
```

    1

```python
backend.provider()
```

    <AccountProvider for IBMQ(hub='ibm-q', group='open', project='main')>

```python
provider = IBMQ.load_account()
```

    ibmqfactory.load_account:WARNING:2022-01-26 14:34:47,725: Credentials are already in use. The existing account in the session will be replaced.

```python
backend = provider.backends()[1]
```

```python
from qiskit import qiskit
qc=QuantumCircuit(2,2)

qc.h(0)
qc.cx(0,1)
qc.measure([0,1],[0,1])

job=qiskit.execute(qc,backend)
job.status()
job.status()
job.result().get_counts()
```

    Traceback (most recent call last):
      Input In [39] in <module>
        job=qiskit.execute(qc,backend)
      File /opt/conda/lib/python3.8/site-packages/qiskit/execute_function.py:294 in execute
        experiments = transpile(
      File /opt/conda/lib/python3.8/site-packages/qiskit/compiler/transpiler.py:307 in transpile
        transpile_args = _parse_transpile_args(
      File /opt/conda/lib/python3.8/site-packages/qiskit/compiler/transpiler.py:584 in _parse_transpile_args
        timing_constraints = _parse_timing_constraints(backend, timing_constraints, num_circuits)
      File /opt/conda/lib/python3.8/site-packages/qiskit/compiler/transpiler.py:1111 in _parse_timing_constraints
        timing_constraints = getattr(backend.configuration(), "timing_constraints", {})
    AttributeError: 'list' object has no attribute 'configuration'

    Use %tb to get the full traceback.

<button class='button iqx-button'>Search for solution online</button>

```python
backend.configuration()
```

    <qiskit.providers.models.backendconfiguration.PulseBackendConfiguration at 0x7fce9122bfd0>

```python
backend
```

    VBox(children=(HTML(value="<h1 style='color:#ffffff;background-color:#000000;padding-top: 1%;padding-bottom: 1…

    <IBMQBackend('ibmq_armonk') from IBMQ(hub='ibm-q', group='open', project='main')>

```python

```

26 Running on a Real Quantum Computer with IBMQ.ipynb


```python
%matplotlib inline
import numpy as np
import pandas as pd
import matplotlib as mpl
import matplotlib.pylab as plt
import sympy
import seaborn as sns
import datetime as dt
```

```python
taxi = pd.read_csv('../../dss7b5-nyctaxi_JSW/train.csv')
test = pd.read_csv('../../dss7b5-nyctaxi_JSW/test.csv')
sample_submission = pd.read_csv('../../dss7b5-nyctaxi_JSW/sample_submission.csv')
taxi["count"] = 1
```

### Data Understanding

```python
print('taxi : {} rows, {} columns.'.format(taxi.shape[0], taxi.shape[1]))
print('test : {} rows, {} columns.'.format(test.shape[0], test.shape[1]))
```

    taxi : 1458644 rows, 12 columns.
    test : 625134 rows, 9 columns.

```python
taxi.tail(2)
```

      id
      vendor_id
      pickup_datetime
      dropoff_datetime
      passenger_count
      pickup_longitude
      pickup_latitude
      dropoff_longitude
      dropoff_latitude
      store_and_fwd_flag
      trip_duration
      count

      1458642
      id2714485
      1
      2016-01-05 15:56:26
      2016-01-05 16:02:39
      1
      -73.982079
      40.749062
      -73.974632
      40.757107
      N
      373
      1

      1458643
      id1209952
      1
      2016-04-05 14:44:25
      2016-04-05 14:47:43
      1
      -73.979538
      40.781750
      -73.972809
      40.790585
      N
      198
      1

```python
test.tail(2)   # drop-off_datetime, trip_duration excluded
```

      id
      vendor_id
      pickup_datetime
      passenger_count
      pickup_longitude
      pickup_latitude
      dropoff_longitude
      dropoff_latitude
      store_and_fwd_flag

      625132
      id1384355
      1
      2016-01-01 00:00:28
      1
      -73.976501
      40.733562
      -73.854263
      40.891788
      N

      625133
      id0621643
      2
      2016-01-01 00:00:22
      2
      -73.981850
      40.716881
      -73.969330
      40.769379
      N

```python

```

```python

```

### Data 뽑기 전 사전작업

```python
# pd.to_datetime을 해줘야 pickup_datetime coulmn의 data type이 datetime으로 됨.
pickup_datetime_dt = pd.to_datetime(taxi["pickup_datetime"])
dropoff_datetime_dt = pd.to_datetime(taxi["dropoff_datetime"])
```

```python
# passenger_count, trip_duration, count를 포함하고, pickup_datetime의 data로만 이루어진 dataframe(taxi_df1)을 만든다.
taxi_df1 = taxi.loc[:, ["passenger_count","trip_duration", "count"]]
taxi_df1["pickup_datetime"] = pickup_datetime_dt
```

```python
# pickup datetime 중 year, month, day
taxi_df1.loc[:, "pickup_date"] = taxi_df1["pickup_datetime"].dt.date
# pickup datetime 중 month만 가져와서 새로 column을 만듦.
taxi_df1.loc[:, "pickup_month"] = taxi_df1["pickup_datetime"].dt.month
# pickup datetime 중 hour만 가져와서 새로 column을 만듦.
taxi_df1.loc[:, "pickup_hour"] = taxi_df1["pickup_datetime"].dt.hour
# pickup datetime 중 요일만 가져와서 새로 column을 만듦.
# 월요일은 0이고, 일요일은 6임.
taxi_df1.loc[:, "pickup_weekday"] = taxi_df1["pickup_datetime"].dt.weekday
```

```python
taxi_df1.tail()
```

      passenger_count
      trip_duration
      count
      pickup_datetime
      pickup_date
      pickup_month
      pickup_hour
      pickup_weekday

      1458639
      4
      778
      1
      2016-04-08 13:31:04
      2016-04-08
      4
      13
      4

      1458640
      1
      655
      1
      2016-01-10 07:35:15
      2016-01-10
      1
      7
      6

      1458641
      1
      764
      1
      2016-04-22 06:57:41
      2016-04-22
      4
      6
      4

      1458642
      1
      373
      1
      2016-01-05 15:56:26
      2016-01-05
      1
      15
      1

      1458643
      1
      198
      1
      2016-04-05 14:44:25
      2016-04-05
      4
      14
      1

```python
### 승객수 / 승차시간 / 승차요일 간의 관계 파악을 위한 표 작성

taxi_df2 = pd.DataFrame(taxi_df1, columns=["passenger_count", "pickup_hour", "pickup_weekday", "trip_duration", "count"])
taxi_df2 = taxi_df1.groupby(["passenger_count", "pickup_hour", "pickup_weekday"]).size().reset_index(name='count')
taxi_df2.tail()
```

      passenger_count
      pickup_hour
      pickup_weekday
      count

      1055
      7
      10
      1
      1

      1056
      7
      19
      5
      1

      1057
      7
      22
      6
      1

      1058
      8
      1
      4
      1

      1059
      9
      8
      4
      1

```python

```

### EDA - 승객 수  data

```python
### 승객수 - Max / Min
```

```python
print(taxi_df2["passenger_count"].min())
print(taxi_df2["passenger_count"].max())
```

    0
    9

```python

```

```python
### 승객수의 비중 - 승객수별 id수 구하기
```

```python
passenger_num_1 = taxi_df2.loc[:, ["passenger_count", "count"]]
passenger_num_1.groupby("passenger_count").sum()
```

      count

      passenger_count

      0
      60

      1
      1033540

      2
      210318

      3
      59896

      4
      28404

      5
      78088

      6
      48333

      7
      3

      8
      1

      9
      1

```python
#### 승객수와 승차시점 관계 (pickup_hour)
```

```python
passenger_pivot1 = taxi_df2.pivot_table(values="count", index=["passenger_count"], columns=["pickup_hour"], aggfunc=np.sum)
passenger_pivot1.fillna(value=0)
```

      pickup_hour
      0
      1
      2
      3
      4
      5
      6
      7
      8
      9
      ...
      14
      15
      16
      17
      18
      19
      20
      21
      22
      23

      passenger_count

      0
      5.0
      8.0
      2.0
      1.0
      2.0
      5.0
      1.0
      0.0
      1.0
      1.0
      ...
      3.0
      1.0
      3.0
      2.0
      1.0
      6.0
      3.0
      4.0
      2.0
      5.0

      1
      36342.0
      26335.0
      19095.0
      14320.0
      11067.0
      11375.0
      26004.0
      42513.0
      50421.0
      50641.0
      ...
      52323.0
      50471.0
      44850.0
      53338.0
      63722.0
      63210.0
      58699.0
      57829.0
      54589.0
      47571.0

      2
      8809.0
      6060.0
      4413.0
      3264.0
      2197.0
      1740.0
      3370.0
      6083.0
      7580.0
      7674.0
      ...
      10689.0
      10616.0
      9814.0
      11913.0
      13492.0
      13788.0
      12864.0
      13546.0
      13521.0
      11465.0

      3
      2384.0
      1893.0
      1370.0
      1007.0
      700.0
      480.0
      830.0
      1640.0
      2172.0
      2174.0
      ...
      3067.0
      3121.0
      2842.0
      3417.0
      3896.0
      3908.0
      3520.0
      3768.0
      3671.0
      3188.0

      4
      1186.0
      867.0
      674.0
      440.0
      349.0
      192.0
      430.0
      702.0
      995.0
      985.0
      ...
      1524.0
      1398.0
      1329.0
      1491.0
      1786.0
      1829.0
      1786.0
      1806.0
      1828.0
      1595.0

      5
      2947.0
      2119.0
      1550.0
      1163.0
      905.0
      662.0
      1531.0
      2772.0
      3517.0
      3754.0
      ...
      3992.0
      3774.0
      3300.0
      3957.0
      4898.0
      4757.0
      4610.0
      4626.0
      4486.0
      3797.0

      6
      1575.0
      1288.0
      868.0
      700.0
      572.0
      548.0
      1082.0
      1890.0
      2366.0
      2434.0
      ...
      2694.0
      2430.0
      2175.0
      2365.0
      2805.0
      2809.0
      2590.0
      2606.0
      2394.0
      2164.0

      7
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      ...
      0.0
      0.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0
      1.0
      0.0

      8
      0.0
      1.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      ...
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0

      9
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      1.0
      0.0
      ...
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0

10 rows × 24 columns

```python
plt.figure(figsize = (30,30))
sns.heatmap(passenger_pivot1, cmap="YlGnBu", annot=True, fmt="")
```

```python

```

```python
#### 승객수와 승차시점 관계 (pickup_weekday)
```

```python
passenger_pivot2 = taxi_df2.pivot_table(values="count", index=["passenger_count"], columns=["pickup_weekday"], aggfunc=np.sum)
passenger_pivot2.fillna(value=0)
```

      pickup_weekday
      0
      1
      2
      3
      4
      5
      6

      passenger_count

      0
      10.0
      3.0
      4.0
      8.0
      10.0
      12.0
      13.0

      1
      135346.0
      147704.0
      153040.0
      158543.0
      158773.0
      147889.0
      132245.0

      2
      25858.0
      26601.0
      27711.0
      29380.0
      31930.0
      37094.0
      31744.0

      3
      7278.0
      7429.0
      7779.0
      8061.0
      9085.0
      10778.0
      9486.0

      4
      3362.0
      3355.0
      3521.0
      3619.0
      4437.0
      5490.0
      4620.0

      5
      9543.0
      10899.0
      11212.0
      11692.0
      11881.0
      12068.0
      10793.0

      6
      6021.0
      6757.0
      6869.0
      7271.0
      7415.0
      7536.0
      6464.0

      7
      0.0
      1.0
      0.0
      0.0
      0.0
      1.0
      1.0

      8
      0.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0

      9
      0.0
      0.0
      0.0
      0.0
      1.0
      0.0
      0.0

```python
sns.heatmap(passenger_pivot2, cmap="YlGnBu", annot=True, fmt="")
```

```python

```

```python
### 승객수와 운행시간 관계 (trip_duration) - 승객수가 많을 수록 duration이 길어질 것이다. (장기)
```

```python
passenger_num_1 = taxi_df1.loc[:, ["passenger_count", "trip_duration"]]
passenger_num_1.groupby("passenger_count").mean()
```

      trip_duration

      passenger_count

      0
      1718.433333

      1
      930.399753

      2
      1005.458335

      3
      1028.236276

      4
      1053.529749

      5
      1070.232174

      6
      1061.355223

      7
      19.666667

      8
      104.000000

      9
      560.000000

```python

```

```python
### EDA 결과
- 승객수 Max 9인, Min 0인 : 0인, 7인이상 삭제 검토 (총 65건)
- 승객수 지표 : 1인 승차객이 전체 약 146만 건 중 103만 건의 비중, 2인 승차객 21만건 / 5인 / 3인 순
- 승객수&승차시각 : 오전 8시 이후 1인 승차객 중심으로 전 시간대 고른 분포
- 승객수&승차요일 : 전반적으로 고른 분포 , 전반적으로 주말보다 수~금 승차건수가 높음
- 승객수&운행시간 : 1인 승객이 평균 1~2분정도 짧게 운행. 전반적으로 1000~1100초 (16분대), 탑승객 없는 택시 유일하게 30분 수준 (공항으로 이동?)
```