# PyTorch tutorial の内容について(3)

- date: 2020-0722
- author: 浅川伸一

[https://github.com/pytorch/tutorials/tree/master/beginner_source/nlp](https://github.com/pytorch/tutorials/tree/master/beginner_source/nlp) を見ると
PyTorch で 自然言語処理を行う場合のチュートリアルは以下とおりである

# Deep Learning for NLP with Pytorch

1. [pytorch_tutorial.py](https://github.com/pytorch/tutorials/blob/master/beginner_source/nlp/pytorch_tutorial.py):
	[PyTorch 入門 Introduction to PyTorch](https://pytorch.org/tutorials/beginner/nlp/pytorch_tutorial.html)

2. [deep_learning_tutorial.py](https://github.com/pytorch/tutorials/blob/master/beginner_source/nlp/deep_learning_tutorial.py):
	[PyTorch による深層学習 Deep Learning with PyTorch](https://pytorch.org/tutorials/beginner/nlp/deep_learning_tutorial.html)

3. [word_embeddings_tutorial.py](https://github.com/pytorch/tutorials/blob/master/beginner_source/nlp/word_embeddings_tutorial.py):
	[単語埋め込み:語彙的意味の符号化 Word Embeddings: Encoding Lexical Semantics](https://pytorch.org/tutorials/beginner/nlp/word_embeddings_tutorial.html)

4. [sequence_models_tutorial.py]((https://github.com/pytorch/tutorials/blob/master/beginner_source/nlp/sequence_models_tutorial.py):
	[系列モデルと LSTM Sequence Models and Long-Short Term Memory Networks](https://pytorch.org/tutorials/beginner/nlp/sequence_models_tutorial.html)

5. [advanced_tutorial.py]((https://github.com/pytorch/tutorials/blob/master/beginner_source/nlp/advanced_tutorial.py):
	[動的意思決定と双方向 LSTM 条件付き確率場 Advanced: Making Dynamic Decisions and the Bi-LSTM CRF](https://pytorch.org/tutorials/beginner/nlp/advanced_tutorial.html)

以下では，このうちの 3 について解説している。

```python
%matplotlib inline
```

# 単語埋め込みモデル 語彙的意味の符号化

単語埋め込みモデルとは，実数ベクトルであり、語彙中の単語毎に 1 つのベクトルが割り当てられます。
自然言語処理 (NLP: Natural Language Processings) では，ほとんどの場合，処理すべき特徴は単語であることが多いです。
ではコンピュータではどのように単語を表現すればよいのでしょうか？
アスキー文字の表現を保存することはできます。
ですが，文字はその単語が何であるかを教えてくれるだけです。
その単語の意味についてはは，さほど意味を持ちません。
(接辞情報から品詞を導き出すことは可能でしょう。
大文字化してある単語などからその語彙特性を導き出すことも可能でしょう。
ですが，そのような特徴にはあまり意味がありません)。
さらに、どのような意味でこれらの表現を組み合わせることができるのでしょうか？
ニューラルネットワークからの出力は、入力が $|V|$ 次元となります。
(訳注: NLP では語彙辞書に登録されている語彙数を表す表記として $||$ を用いる習慣がある)
$V$ は語彙辞書を表します。ですが，出力は数次元であることが多いです。
例えばラベルの一部だけを予測する場合などです。
(訳注: NLP では出現頻度の少ない単語を OOV (Out Of Vocabulary) として一括してラベル `<unk>` などを割り当てて
しまうことが行われる。
これを OOV 問題などという。
`<unk>` ラベルを割り当てることにより，極端に学習時間が必要な低頻度語の処理を無視して効率向上を意図する場合がある。
同様にして，具体的な数字なども 一括して同一ラベル，例えば '<num>` あるいは `<digit>` などにする。
実数以上の数概念は概念上無限に存在するので，各数値にラベルを割り振ることは効率的でないことから，
このような前処理が行われる
)
どうやって巨大な次元空間から小さな次元空間に移動させのでしょうか？

アスキー表現の代わりに、ワンホットエン表現を使うのはどうでしょうか？
つまり $w$ という単語を以下のようにします:

\begin{align}\overbrace{\left[ 0, 0, \dots, 1, \dots, 0, 0 \right]}^\text{|V| elements}\end{align}

ここで 1 は単語 $w$ に固有の場所になります。
他の単語はどこか他の場所に 1 があり，それ以外の場所には 0 があります。
(訳注: すなわち ワンホットベクトルとは，いずれか一つの要素だけが 1 で他の要素はすべて 0 であるベクトルを言う。
ビショップの有名な教科書 PAML までは 1-of-k 表現と読んでいた。
だが最近では，ワンホット表現と呼ぶようになってきている。)

このワンホット表現には，その巨大さ以外にも大きな欠点があります。
この表現は基本的に、すべての単語を互いに関係のない独立した実体として扱います。

私たちが本当に必要としているのは、単語間の *類似性* という概念です。
なぜでしょうか？例を見てみましょう。

言語モデルを構築しているとしましょう。次のような文章を見た (言語モデルに与えられた) とします。

- その数学者は店に駆け込んだ。
- その物理学者は店に駆け込んだ。
- その数学者はその未解決問題を解いた。

ここで，学習データ中には存在しない新しい文が出てきたとしましょう

- その物理学者はその未解決問題を解いた。

我々人間の持っている言語モデルでは，この文でも問題ないかもしれません。
ですがが，次の 2 つの事実を使うことができればもっと良いのではないでしょうか？

- 我々は，ある文の中で数学者と物理学者が同じ役割を果たしているのを見たことがある。どういうわけか，彼らは意味的な関係を持っている。
- 我々は，今，物理学者を見ているのと同じように，この新しい文中で数学者を同じ役割で見てきた，と推論して，
物理学者が実はこの新しい見たことのない文の中で同じ役割を果たしている

これが類似性の概念の意味です。
単に類似した正書法表現を持つことではなく，意味的な類似性を意味しています。
これは，見たことのあるものと見ていないものの間の点を結びつけることで，言語データの乏しさに対抗するための技法です。
この例はもちろん、言語学的な基本的な仮定に依存しています：類似した文脈に登場する単語は、意味的に互いに関連しているということです。これは [分布仮説](https://en.wikipedia.org/wiki/Distributional_semantics) と呼ばれています。

### 密な単語埋め込み表現を得る

この問題を解決するにはどうすればよいのでしょうか。
すなわち，どのようにすれば，実際に単語の意味的類似性を符号化することができるのでしょうか？
意味的な属性を考えてみましょう。
例えば，数学者も物理学者も走ることができるので，「走ることができる」という意味的属性に高いスコアを与えることができるかもしれません。
他の属性について考えてみて，それらの属性に共通する単語にどのようなスコアをつけるかを想像してみてください。

各属性に次元が割当可能であれば，各単語には次のようなベクトルを付与できるでしょう:

\begin{align}q_\text{mathematician} = \left[ \overbrace{2.3}^\text{can run},
   \overbrace{9.4}^\text{likes coffee}, \overbrace{-5.5}^\text{majored in Physics}, \dots \right]\end{align}

\begin{align}q_\text{physicist} = \left[ \overbrace{2.5}^\text{can run},
   \overbrace{9.1}^\text{likes coffee}, \overbrace{6.4}^\text{majored in Physics}, \dots \right]\end{align}

こうすれば，単語間の類似性の尺度を得ることができます:

\begin{align}\text{Similarity}(\text{physicist}, \text{mathematician}) = q_\text{physicist} \cdot q_\text{mathematician}\end{align}

一般には，長さで正規化します:

\begin{align}\text{Similarity}(\text{physicist}, \text{mathematician}) = \frac{q_\text{physicist} \cdot q_\text{mathematician}}
   {\| q_\text{physicist} \| \| q_\text{mathematician} \|} = \cos (\phi)\end{align}

ここで $\phi$ は 2 つのベクトルの間の角度です。
このようにして 極端に似ている単語 (埋め込みベクトルが同じ方向を向いている単語） は 類似度が 1 になります。
極めて似ていない単語は類似度が -1 になります。

本節冒頭にある 疎なワンホットベクトルは，我々が定義したこれらの新しいベクトルの特殊な場合と考えることができます。
これらの新しいベクトルは *密 dense* です。
すなわち，ベクトルの各要素は (典型的には) 非ゼロであることを意味します。

しかし、これらの新しいベクトルには大きな問題があります:
類似度を決定するのに関連する何千もの異なる意味属性を考えることができます。
一体どうやって異なる属性の値を設定するのでしょうか？
ディープラーニングの考え方の中心にあるのは，プログラマが自分で特徴を設計するのではなく，
ニューラルネットワークが特徴表現を学習するということです。
では、単語の埋め込みをモデルのパラメータにして、学習中に更新するのはどうでしょうか？
これをまさに我々は行おうとしています。
ネットワークが原理的に学習できるいくつかの *潜在意味属性 latent semantic attributes* を持つことになります。
単語の埋め込みベクトルは，おそらく解釈できないことに注意してください。
つまり，上記の手作りのベクトルでは，数学者と物理学者はコーヒーが好きという点で似ていることがわかります。
ですが，ニューラルネットワークに埋め込みベクトルを学習させて，数学者と物理学者の両方が 2 次元で大きな値を持っていることがわかったとしても，それが何を意味するのかはわかりません。
彼らは何らかの潜在的な意味的次元で似ているが、これはおそらく我々には何の解釈も与えません。

要約すると、**単語埋め込みモデルとは 単語の *意味* を表現したモデルである。このモデルは当該課題に関連する意味情報を効率的に符号化する**。
他のものも埋め込むことができます: 音声タグの一部、構文解析木など，何でもいいのです。
特徴埋め込みモデルのアイデアは，この分野の中心的なものです。

### Pytorch での単語埋め込みモデル

作業例と演習に入る前に，Pytorch や一般的な深層学習プログラミングでの埋め込みモデル の使用方法について簡単な覚書を記しておきます。
ワンホットベクトルを作るときに 各単語にユニークなインデックスを定義したのと同じように，単語埋め込みモデルを使うときにも各単語にインデックスを定義する必要があります。
これらはルックアップテーブルへのキーとなります。
つまり、埋め込み表現は $|V| \times D$ 行列として格納されます。
ここで $D$ は埋め込みの次元数，インデックス $i$ が割り当てられた単語は，その埋め込みが行列の $i$' 番目の行に格納されます。
以下のコードでは，単語からインデックスへのマッピングは word\_to\_ixと いう名前の辞書になっています。

埋め込みを使用するためのモジュールは ``torch.nn.Embedding`` で 2 つの引数をとります。

このテーブルにインデックスを作成するには ``torch.LongTensor`` を使用しなければなりません
(インデックスは浮動小数点ではなく整数)。

```python
# Author: Robert Guthrie

import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim

torch.manual_seed(1)
```

    <torch._C.Generator at 0x7f4d769cf5d0>

```python
word_to_ix = {"hello": 0, "world": 1}
embeds = nn.Embedding(2, 5)  # 2 つの単語を 5 次元の埋め込みモデルとして表現
lookup_tensor = torch.tensor([word_to_ix["hello"]], dtype=torch.long)
hello_embed = embeds(lookup_tensor)
print(hello_embed)
```

    tensor([[ 0.6614,  0.2669,  0.0617,  0.6213, -0.4519]],
           grad_fn=<EmbeddingBackward>)

### 例: N-グラム言語モデル

N-Gram 言語モデルでは，与えられた単語列 $w$ を次のように表現します:

\begin{align}P(w_i | w_{i-1}, w_{i-2}, \dots, w_{i-n+1} )\end{align}

ここで $w_i$ は系列位置 $i$ の単語を表します。

本例では，訓練データに基づいて損失関数を計算し，誤差逆伝播法でパラメータを更新します

```python
CONTEXT_SIZE = 2
EMBEDDING_DIM = 10
# シェイクスピアのソネット 2 を使います
test_sentence = """When forty winters shall besiege thy brow,
And dig deep trenches in thy beauty's field,
Thy youth's proud livery so gazed on now,
Will be a totter'd weed of small worth held:
Then being asked, where all thy beauty lies,
Where all the treasure of thy lusty days;
To say, within thine own deep sunken eyes,
Were an all-eating shame, and thriftless praise.
How much more praise deserv'd thy beauty's use,
If thou couldst answer 'This fair child of mine
Shall sum my count, and make my old excuse,'
Proving his beauty by succession thine!
This were to be new made when thou art old,
And see thy blood warm when thou feel'st it cold.""".split()
# 訳注: 徒然草 冒頭へ変更
test_sentence = """つれづれ なる まま に 日暮らし 硯 に むかひ て 心 に うつり ゆく よしなし ごと を そこはかとなく 書き つくれ ば あやしう こそ ものぐるほし け れ""".split()
# 入力をトークン化する必要がありますが、今のところは無視して
# タプルのリストを作成します。各タプルは ([ word_i-2, word_i-1 ], target word) です.
trigrams = [([test_sentence[i], test_sentence[i + 1]], test_sentence[i + 2])
            for i in range(len(test_sentence) - 2)]
# 最初の 3 つを表示します。どうなっているかを確認してください
print(trigrams[:3])

vocab = set(test_sentence)
word_to_ix = {word: i for i, word in enumerate(vocab)}

class NGramLanguageModeler(nn.Module):

    def __init__(self, vocab_size, embedding_dim, context_size):
        super(NGramLanguageModeler, self).__init__()
        self.embeddings = nn.Embedding(vocab_size, embedding_dim)
        self.linear1 = nn.Linear(context_size * embedding_dim, 128)
        self.linear2 = nn.Linear(128, vocab_size)

    def forward(self, inputs):
        embeds = self.embeddings(inputs).view((1, -1))
        out = F.relu(self.linear1(embeds))
        out = self.linear2(out)
        log_probs = F.log_softmax(out, dim=1)
        return log_probs

losses = []
loss_function = nn.NLLLoss()
model = NGramLanguageModeler(len(vocab), EMBEDDING_DIM, CONTEXT_SIZE)
optimizer = optim.SGD(model.parameters(), lr=0.001)

for epoch in range(10):
    total_loss = 0
    for context, target in trigrams:

        # ステップ 1. モデルに渡す入力を準備します (すなわち、単語を整数インデックスに
        # 変換してテンソルにします)
        context_idxs = torch.tensor([word_to_ix[w] for w in context], dtype=torch.long)

        # ステップ2. PyTorch は勾配を蓄積することを思い出してください
        # 新しいインスタンスに渡す前に，古いインスタンスの勾配を ゼロで
        # 初期化する必要があります。
        model.zero_grad()

        # ステップ 3. データの前向き伝播，次単語の対数確率を計算します
        log_probs = model(context_idxs)

        # ステップ 4. 損失関数の計算します (PyTorch では目標単語をテンソル化して渡します)
        loss = loss_function(log_probs, torch.tensor([word_to_ix[target]], dtype=torch.long))

        # ステップ 5. 誤差勾配の逆向き伝播
        loss.backward()
        optimizer.step()

        # tensor.item() を呼び出すことで 要素が 1 であるテンソルから数値を取り出す
        total_loss += loss.item()
    losses.append(total_loss)
print(losses)  # 損失値は訓練の繰り返し時に減少する
```

    [(['つれづれ', 'なる'], 'まま'), (['なる', 'まま'], 'に'), (['まま', 'に'], '日暮らし')]
    [73.11203050613403, 72.59552907943726, 72.08381295204163, 71.57616806030273, 71.0731291770935, 70.57366609573364, 70.07831168174744, 69.58672142028809, 69.09947919845581, 68.61528730392456]

### 演習 単語埋め込みベクトルの計算。CBOW モデル

NLP のディープラーニングでは CBOW（Continuous Bag-of-Word）モデルがよく使われています。
CBOW モデルは，対象となる単語の前の数語と後の数語の文脈を与えられた単語を予測しようとするモデルです。
CBOW が逐次的ではなく、確率的である必要がないため、言語モデルとは異なります。
通常 CBOW は単語埋め込みを素早く訓練するために使用されます。
この単語埋め込みモデルは，より複雑なモデルの埋め込み表現を初期化するために使用さます。
通常 このことを *事前埋め込み pretraining embeddings* と呼びます。
これにより，ほとんどの場合，パフォーマンスが数パーセント向上します。

CBOW モデルは次のとおりです:
目標誤 $w_i$ と 目標語の前後を表す内容語 $C$ が幅 $N$ のウィンドウで与えられます。w_{i-1}, \dots, w_{i-N}$ and $w_{i+1}, \dots, w_{i+N}$, CBOW モデルは以下の目的関数を最小化させます:

\begin{align}-\log p(w_i | C) = -\log \text{Softmax}(A(\sum_{w \in C} q_w) + b)\end{align}

ここで $q_w$ は 単語 $w$ の埋め込みベクトルです。

このモデルを PyTorc で実装するには以下のクラスを必要とします。
ヒントをいくつか以下に:

* どのようなパラメータを定義する必要があるか考えてください。
* 各操作がどのような形を想定しているかを確認してください。reshape が必要な場合は .view() を使用してください。

```python
CONTEXT_SIZE = 2  # 前後の 2 単語を内容語とします
raw_text = """We are about to study the idea of a computational process.
Computational processes are abstract beings that inhabit computers.
As they evolve, processes manipulate other abstract things called data.
The evolution of a process is directed by a pattern of rules
called a program. People create programs to direct processes. In effect,
we conjure the spirits of the computer with our spells.""".split()

# By deriving a set from `raw_text`, we deduplicate the array
vocab = set(raw_text)
vocab_size = len(vocab)

word_to_ix = {word: i for i, word in enumerate(vocab)}
data = []
for i in range(2, len(raw_text) - 2):
    context = [raw_text[i - 2], raw_text[i - 1],
               raw_text[i + 1], raw_text[i + 2]]
    target = raw_text[i]
    data.append((context, target))
print(data[:5])

class CBOW(nn.Module):

    def __init__(self):
        pass

    def forward(self, inputs):
        pass

# create your model and train.  here are some functions to help you make
# the data ready for use by your module

def make_context_vector(context, word_to_ix):
    idxs = [word_to_ix[w] for w in context]
    return torch.tensor(idxs, dtype=torch.long)

make_context_vector(data[0][0], word_to_ix)  # example
```

    [(['We', 'are', 'to', 'study'], 'about'), (['are', 'about', 'study', 'the'], 'to'), (['about', 'to', 'the', 'idea'], 'study'), (['to', 'study', 'idea', 'of'], 'the'), (['study', 'the', 'of', 'a'], 'idea')]

    tensor([ 9, 30,  3,  1])

```python

```


# "WxBS: Relaunching challenging benchmark for Image Matching"
> "And I mean really challenging"

- toc: false
- image: images/loftr_match.png
- branch: master
- badges: true
- comments: true
- hide: false
- search_exclude: false

## The hardest image matching benchmark

Previously we have discussed [what is WxBS in general](https://ducha-aiki.github.io/wide-baseline-stereo-blog/2021/01/09/wxbs-in-simple-terms.html), [how to match images from a really different viewpoints](https://ducha-aiki.github.io/wide-baseline-stereo-blog/2020/08/06/affine-view-synthesis.html) and also took a look into [Image Matching Challenge 2020](https://ducha-aiki.github.io/wide-baseline-stereo-blog/2021/05/14/IMC2020-competition-recap.html). Let's now speak about how to measure progress on images, which are really hard to match even to human?

For example, which have been taked in day-vs-night, years apart AND from different camera positions? One cannot register them to some 3d model, because they are too hard. And there is no 3D model either.

Well, in that case one could go back to basics and *handlabel* the correspondences between the images. Such annotation is unlikely to be very precise, but it is going to be quite robust, if the labeler is careful.

That is what I have done 6 years ago and published such a dataset at [BMVC 2015](http://www.bmva.org/bmvc/2015/papers/paper012/index.html) together with a benchmark of popular image matchers of that time. None of them was able to successfully match even a fraction of the WxBS dataset.

However, as I was quite inexpereinced in product packaging, as well as in software engineering, the benchmark existed as dataset + bunch of bash and Matlab scripts. It is needless to say, that nobody evaluated their algorithms on it.
That made me sad, however, I never found time to make this benchmark easy to use.
Well, now I did it and want to share the evaluation of the recent agorithms on such a hard data.

## Not that fast! Problem with a fundamental matrix and how to solve them

OK, we have ground truth correspondences. Now what? Ideally one can estimate epipolar geometry from as little, as 7 correspondences, but in practice? No. The problem is that there could be many relative camera poses, which are consistent with the correspondences, especially when correspondences lie in plane and the camera calibration is unknown.

Let's consider a simple example: we have 13 correspondences and they seems to be correct. Are they enough for finding good epipolar geometry? This question can be answered via leave-one-out validation procedure, described in [VSAC paper](https://arxiv.org/abs/2106.10240). We remove a single correspondence, estimate a fundamental matrix via DLT algorithm on the rest of the correspondences and see how the estimated epipolar geometry is changed (or not).

As you can see on the animation below, the epipolar geometry estimation from our 13 correspondences is not stable and removing any of them results in noticable change.

So, if even manually labeled corresposndences are not good enough, what can be done?

First, simply add more correspondences, which are spread across image AND depth levels as even as possible. That is what I did recently, in order to make WxBS labeling better. However, it is not always possible -- sometimes you have lots of correspondence on "faraway background plane" and very little somewhere on foreground, see next picture.

What else can be done? We can go other way round and *not provide any ground truth fundamental matrix at all*.
Instead, we ask methods, which we evaluate, to give us such matrix *F_est*, then check, how many ground truth correspondences are consident with it.

The bad thing about it, is that unless all 100% GT correspondences are consistent with *F_est*, *F_est* is completely wrong in terms of camera pose accuracy. So, such a binary signal on a very challenging pairs is not that useful.
The good thing about it, is that if, say, 50% GT correspondences are consistent with *F_est*, it means that detected (not GT) correspondences *on the part of the image* are correct (e.g. on some dominant plane) and at least there we are OK.

If you are interested in more mathematical definition, here it is:

The recall on ground truth correspondences $C_i$ of image pair $i$ and for geometry model $\mathbf{M}_i$ is computed as a function of a threshold $\theta$

\begin{equation}
 \mathrm{r}_{i,\mathbf{M}_i}(\theta) = \frac{| \{(\mathbf{u},\mathbf{v}) : (\mathbf{u},\mathbf{v}) \in C_i,  e(\mathbf{M}_i,\mathbf{u},\mathbf{v}) < \theta\}|}{| C_i |}
% \mathrm{r}_{i,\mathbf{M}_i}(\theta) = \frac{| \{(\mathbf{u}_i,\mathbf{v}_i) : (\mathbf{u}_i,\mathbf{v}_i) \in C_i,  e(\mathbf{M}_i,\mathbf{u},\mathbf{v}) < \theta\}|}{| C_i |}
\end{equation}

using [symmetric epipolar distance](https://kornia.readthedocs.io/en/latest/geometry.epipolar.html#kornia.geometry.epipolar.symmetrical_epipolar_distance).

## Let's evaluate!

First I will show you how to run the WxBS benchmark yourself and then present the results I got.  Benchmark is available from pip:

```python
!pip install wxbs_benchmark
!pip install kornia_moons
```

Now we will write a simple function using OpenCV RootSIFT and [MAGSAC++](https://arxiv.org/abs/1912.05909). RootSIFT is a gold standard handcrafted local feature and MAGSAC++ is a state-of-the-art RANSAC, which is not sensitive to an inlier threshold, as [my recent benchmark](https://ducha-aiki.github.io/wide-baseline-stereo-blog/2021/05/17/OpenCV-New-RANSACs.html) shows. We will match then using mutual second nearest ratio, as implemented in [kornia](https://github.com/kornia/kornia)

```python
import matplotlib.pyplot as plt
import numpy as np
import cv2
import torch
import kornia.feature as KF
from kornia_moons.feature import *
from wxbs_benchmark.dataset import *
from wxbs_benchmark.evaluation import *
from tqdm import tqdm

def sift2rootsift(desc):
    desc /= desc.sum(axis=1, keepdims=True) + 1e-8
    desc = np.sqrt(desc)
    return desc

def estimate_F_WithRootSIFT(img1, img2):
    det = cv2.SIFT_create(8000, contrastThreshold=-10000, edgeThreshold=10000)
    kps1, descs1 = det.detectAndCompute(img1, None)
    kps2, descs2 = det.detectAndCompute(img2, None)
    descs1 = sift2rootsift(descs1)
    descs2 = sift2rootsift(descs2)
    snn_ratio, idxs = KF.match_smnn(torch.from_numpy(descs1),
                                    torch.from_numpy(descs2), 0.95)
    tentatives = cv2_matches_from_kornia(snn_ratio, idxs)
    src_pts = np.float32([ kps1[m.queryIdx].pt for m in tentatives ]).reshape(-1,2)
    dst_pts = np.float32([ kps2[m.trainIdx].pt for m in tentatives ]).reshape(-1,2)
    F, inlier_mask = cv2.findFundamentalMat(src_pts, dst_pts, cv2.USAC_MAGSAC, 0.25, 0.999, 100000)
    return F

subset = 'test'
dset = WxBSDataset('.WxBS', subset=subset, download=True)

F_results_RootSIFT = []

for pair_dict in tqdm(dset):
    current_F = estimate_F_WithRootSIFT(pair_dict['img1'], pair_dict['img2'])
    F_results_RootSIFT.append(current_F)

result_dict_rootsift, thresholds = evaluate_Fs(F_results_RootSIFT, subset)
```

We can check results for individual image pairs, or just take an average

```python
print(result_dict_rootsift.keys())
plt.figure()
plt.plot(thresholds, result_dict_rootsift['average'], '-x')
plt.ylim([0,1.05])
plt.xlabel('Thresholds')
plt.ylabel('Recall on GT corrs')
plt.grid(True)
plt.legend(['RootSIFT + MAGSAC++'])
```

I have run several popular recent methods in this [Colab](https://colab.research.google.com/drive/1yrCFyEoAc0HyqYCRVvzJDh5kQT2Dc3XA?usp=sharing). The code is very dirty, that is why I don't put it here, and just present results.

## Methods tested

### Local features without learned matching.

[RootSIFT](https://www.robots.ox.ac.uk/~vgg/publications/2012/Arandjelovic12/arandjelovic12.pdf) (ICCV1999 - CVPR-2012), [DoG](https://www.cs.ubc.ca/~lowe/papers/ijcv04.pdf)+[HardNet](https://arxiv.org/pdf/1705.10872.pdf) (NeurIPS 2017), [R2D2](https://arxiv.org/abs/1906.06195) (NeurIPS 2019), [DISK](https://arxiv.org/abs/2006.13566) (NeurIPS 2020).

### Local features with learned matching.

[SuperGlue](https://arxiv.org/abs/1911.11763) (CVPR2020) uses attention-based graph neural network to match SuperPoint local features.

### Dense matching

[PDCNet](https://prunetruong.com/research/pdcnet) (CVPR 2021). On top of pretrained ImageNet coarse-to-fine feature correlation,  PDCNet predicts a dense flow and a confidence for each pixel. Trained on huge dataset with synthetic warps.

[Patch2pix](https://arxiv.org/abs/2012.01909) (CVPR 2021). Similarly to PDCNet, patch2pix is uses pretrained ImageNet features to get the initial corresponcences. Then the matches are progressively refined or rejected with a regressor network.

[DFM](https://arxiv.org/abs/2106.07791) (CVPRW 2021). Like PDCNet and Patch2pix, DFM uses ImageNet-pretrained and frozen VGGNet backbone as dense feature extractor. Unlike them, DFM does not requite any training and relies on simple handcrafted coarse-to-fine matching from conv5 to conv1.

### Dense matching with transformers

[COTR](https://arxiv.org/abs/2103.14167), (ICCV2021). Given the pixel location in a query image, COTR finds a corresponing location in the reference image. It operated on the 16x16 feature map produced by pretrained ImageNet model, which then are fed together with a positional embeddings to a transformer-based regressor.
It is also the slowest method among evaluated - because you have to run it *per each pixel separately*, if want dense correspondences.

[LoFTR](https://zju3dv.github.io/loftr/) (CVPR2021). LoFTR is very similar to Patch2pix, with the difference of using transformer architecture to for both coarse initial matching and consequent refinement.
The second difference, is that, unlike all other methods, which are using ImageNet-pretrained models as a feature backbone, LoFTR trains ResNet18 *from scratch* .

## Results & Discussion

The clear leader is LoFTR, then we have (patch2pix, COTR and SuperGlue) group, followed by (PDCNet, DISK, R2D2). Then DoG-HardNet and RootSIFT, where latter has barely matched couple of image pairs correctly.

I have no idea, why LoFTR is so good, my guesses would be richer training data and training the backbone from scratch, instead of relying on ImageNet feaures. I am also quite surprised by the great performance of the patch2pix, which probably can be improved even more, if the backbone would be at least fine-tuned.

Another surprise is a great performance of the DFM matcher, which is slightly better than PDCNet and worse (but comparable) than COTR. My hypothesis is that all this methods rely on the same ImageNet backbone: if they fail, all the methods fail.

Regarding SuperGlue - one of my guesses is that original SuperPoint features were not trained for different illuminations. Moreover, in some images, there just not enough matching corners -  when I was annotating the correspondences, I have to use also "center of the blobs".

Finally, I would like to that than even excellent LofTR result is far from the ideal -- there are stil tens of pairs, where it fails.

P.S. I would also like to thank Prune Truong and Ufuk Efe who helped me to debug my originally wrong run of the PDCNet and DFM respectively.

### Qualiative examples

Here are some examples of how different methods work on the same image pairs.

### Season + viewpoint

LoFTR

SuperGlue

patch2pix

### Thermal vs visible + viewpoint

LoFTR

SuperGlue

patch2pix

### Season + illumination + viewpoint

LoFTR

SuperGlue

patch2pix

### Illumination + viewpoint change (zoom)

LoFTR

SuperGlue

patch2pix

As you can, see, there are many incorrect correspondences even in those images, where methods are performing well.

Here I will conclude and go to vacation.

In the 2nd part of the post, I would like to evaluate COTR, PDCNet and different dense descriptors directly on the GT correspondences, by giving a point in image A and asking for point in image B.


# Week 1 - Object Detection Assignments

- Use any coding language you want
- If we specifically ask you to code a function, please do not use any available implementations directly from packages.

## Question 1

Part 0: Compare **one-stage object detectors** and **two-stage object detectors**.

Part 1: Why is **Faster-RCNN** slow?

Part 2: Why or how is **YOLO** so damn fast?

Part 3: What is the improvement of **SSD** over **YOLO**?

Write your answer to the cell below. Please try to give a detailed answer for each part.

**Answer**:

## Question 2

**Non-Maximum Supression (NMS)** is used when there are multiple bounding boxes overlapping each other, belonging to the same object.

Here, $\lambda_{nms}$ is a predefined hyperparameter that determines the threshold for "overlapping".

Code NMS.

```python
import cv2
import numpy as np
from google.colab.patches import cv2_imshow

# Download the image of Lenna to your own drive, and change the file path accordingly:
# https://drive.google.com/file/d/1IWm1NzV4_z8hFknClmkun4oNqkHHASQS/view?usp=sharing
image = cv2.imread("lenna.jpg")

cv2.rectangle(image, (119, 149), (427, 388), (0, 255, 0), 2)
cv2.rectangle(image, (152, 218), (397, 371), (0, 255, 0), 2)
cv2.rectangle(image, (197, 174), (382, 440), (0, 255, 0), 2)
cv2.rectangle(image, (183, 190), (378, 400), (0, 255, 0), 2)

cv2_imshow(image)
```

```python
def non_max_suppression(boxes, scores):
  # Write the code for NMS here
  pass
```

```python
boxes = [[119, 149, 427, 388], [152, 218, 397, 371], [197, 174, 382, 440], [183, 190, 378, 400]]
scores = [0.78, 0.92, 0.85, 0.82]

updated_boxes = non_max_suppression(boxes, scores)
```

```python
image = cv2.imread("lenna.jpg")

# Enter the coordinates of the "updated_boxes" below:
# cv2.rectangle(image, (119, 149), (427, 388), (0, 255, 0), 2)

cv2_imshow(image)
```

Can you backpropagate through NMS?

**Bonus:** If the answer is yes, we dare you to code it.

Hint: You can check out some papers on learnable NMS online.

```python
# Code learnable NMS here. You can use any package that you want.
```

## Question 3

This question consists of four parts:

1. Write the code of **Spatial Pyramid Pooling (SPP)** operation.
1. Write the code of **RoIPool** operation.
1. Write the code of **RoIAlign** operation.
1. Compare the three methods in terms of their advantages and disadvantages.

### Part 1: Spatial Pyramid Pooling (SPP)

There is no need for flattening convolution features into 1D and then feed them to fully connected layers anymore! Since fully connected layers only accept fixed-size input, convolutional neural networks also only accepted fixed size input (most commonly $224 \times 224$) for a while.

Spatial Pyramid Pooling (SPP) keeps the spatial information of the feature maps in local spatial bins. The sizes of these bins, and the number of bins are all fixed. Responses of the filters are pooled in each spatial bin. (In the original [paper](https://arxiv.org/abs/1406.4729), authors use max pooling).

Image source: Paper

Code SPP with any framework you want. Your function should take output feature maps of a convolutional layer as input, and should give a fixed size output, whatever the size of the input feature maps.

```python
# A random feature map, belonging to one image:

import numpy as np
np.random.seed(3) # change the seed for different inputs

num_filters = np.random.choice([64, 128, 256])
height = np.random.randint(12, 42)
width = height

feature_maps = np.random.randn(num_filters, height, width)
print(feature_maps.shape)
```

    (256, 36, 36)

```python
def spatial_pyramid_pooling(feature_maps, levels):
  # code for SPP will be written here
  pass
```

If the levels are ```[1, 2, 4]```, and the filter size of the feature maps is $64$, then the output of SPP should be $64 \times 1 + 64 \times 4 + 64 \times 16 = 64 \times 21$

### Part 2: ROI Pooling

Much like SPP, ROI (Region of Interest) Pooling also creates fixed size feature vectors from arbitrary sized convolutional feature maps by applying max pooling to them.

Difference from SPP is that we also have ROI proposals next to the convolutional feature maps, and we only pool from the respective regions of ROIs on the convolutional feature maps for each ROI.

ROI Pooling layer has two inputs:

- A feature map from the previous convolutional layer (just like SPP)
- $N$ ROIs or proposals from the Region Proposal Network. Each ROI has five parameters, first one is the index and the second and the third ones are the coordinates of the top-left and bottom-right of the ROI w.r.t. input.

For every ROI, ROI Pooling gets the corresponding region from the scaled down feature map, and then divides that region to ```pool_width``` x ```pool_height```, then applies max pooling to that region to obtain fixed-length feature vectors for each ROI. "Scaling down" means that if the input is $256 \times 256$ and the feature map is $32 \times 32$, then the scaling factor is $1 / 8$, and the coordinates of the original ROI should be scaled down by a factor of $1 / 8$.

Image source: TowardsDataScience

```python
# You can use the feature maps created for SPP above

# You can use the dummy proposals created here
proposals = [[0., 12., 15., 45., 48.], [1., 10., 10., 67., 212.], [2., 122., 125., 156., 155.], [3., 48., 40., 175., 179.]]
proposals = np.array(proposals)

def roi_pooling(feature_maps, proposals):
  # code for SPP will be written here
  pass
```

If there are $4$ proposals, $256$ channels, and the pooling height and pooling width are $3 \times 3$, then the output should be $4 \times 256 \times 3 \times 3$

### Part 3: RoIAlign

In ROI Pooling, we lose some information because of the quantization from downscaling the ROI to the feature map. Some of the coordinates are rounded to fit the scaled ROI into the feature map since we can not always downscale the ROI to precise integers.

ROI Align is proposed to overcome the loss of information from this quantization. The keyword of ROI Align is **bilinear interpolation**. Check out the figure below to see the process of ROI Align:

Image source: Medium

Given the feature maps, and the proposals at the above cells, write the code for ROI Align to the cell below.

```python
def roi_align(feature_maps, proposals):
  # Code of ROI Align will be written here
  pass
```

### Part 4: Comparison of the Methods

Compare the three different approaches in terms of their advantages and disadvantages. Write your answer to the cell below.

**Answer:**

## Question 4

Why do we use **Intersection over Union (IoU)** as a metric? What happens with scale change? Please elaborate on the example of when you detect a small object wrong vs. when you detect a large object wrong?

Intersection over Union for regions $A$ and $B$ is defined as:

\begin{equation}
J(A, B) = \frac{|A \cap B|}{|A \cup B|}
\end{equation}

Write your answer to the cell below.

**Answer:**

Run the code below for given ground truth boxes and prediction boxes.

```python
# code taken from: https://gist.github.com/meyerjo/dd3533edc97c81258898f60d8978eddc
def bb_intersection_over_union(boxA, boxB):
    # determine the (x, y)-coordinates of the intersection rectangle
    xA = max(boxA[0], boxB[0])
    yA = max(boxA[1], boxB[1])
    xB = min(boxA[2], boxB[2])
    yB = min(boxA[3], boxB[3])

    # compute the area of intersection rectangle
    interArea = abs(max((xB - xA, 0)) * max((yB - yA), 0))
    if interArea == 0:
        return 0
    # compute the area of both the prediction and ground-truth
    # rectangles
    boxAArea = abs((boxA[2] - boxA[0]) * (boxA[3] - boxA[1]))
    boxBArea = abs((boxB[2] - boxB[0]) * (boxB[3] - boxB[1]))

    # compute the intersection over union by taking the intersection
    # area and dividing it by the sum of prediction + ground-truth
    # areas - the interesection area
    iou = interArea / float(boxAArea + boxBArea - interArea)

    # return the intersection over union value
    return iou
```

```python
print('Two boxes with no overlap')
boxA = [0., 0., 10., 10.]
boxB = [12., 12., 22., 22.]

print(bb_intersection_over_union(boxA, boxB))

print('Two boxes with some overlap')
boxA = [0., 0., 12., 12.]
boxB = [6., 6., 20., 20.]

print(bb_intersection_over_union(boxA, boxB))

print('Two boxes with more overlap')
boxA = [0., 0., 12., 12.]
boxB = [2., 2., 13., 13.]

print(bb_intersection_over_union(boxA, boxB))

print('Two boxes with full overlap')
boxA = [0., 0., 12., 12.]
boxB = [0., 0., 12., 12.]

print(bb_intersection_over_union(boxA, boxB))
```

    Two boxes with no overlap
    0
    Two boxes with some overlap
    0.11842105263157894
    Two boxes with more overlap
    0.6060606060606061
    Two boxes with full overlap
    1.0

## Question 5

What is the **classifier imbalance** problem? Why is it relevant for object detection? How is it handled? Please give a detailed answer to the cell below.

Hint: How do you define negative class?

**Answer:**

## Question 6

L2 loss, also known as Mean Squared Error (MSE) can be calculated as:

\begin{equation}
\mathcal{L}_{MSE} = \frac{1}{M} \sum^M_{i=0}(y_i - \hat{y}_i)^2
\end{equation}

where $y_i$ is the $4$-dimensional ground truth bounding box coordinates, and $\hat{y}_i$ is the predicted $4$-dimensional bounding box coordinates.

L1 loss, on the other hand, known as Mean Absolute Error (MAE) can be calculated as following:

\begin{equation}
\mathcal{L}_{MAE} = \frac{1}{M} \sum^M_{i=0}|y_i - \hat{y}_i|
\end{equation}

Finally, the smooth L1 loss, which takes advantage of the strengths of both L1 and L2 loss, is given below:

\begin{equation}
\mathcal{L}_{smooth} = \left\{ \begin{array}{cc}
0.5x^2 &  \hskip 4pt |x| < 1 \\
|x| - 0.5 & \text{otherwise}
\end{array}\right.
\end{equation}

Compare L2 loss vs. smooth L1 loss. Why do we estimate normalized coordinates? Why do we use log for scale?

```python
import numpy as np

def L2_loss(gt_boxes, pred_boxes):
  return np.sum(np.square(gt_boxes-pred_boxes))

def L1_loss(gt_boxes, pred_boxes):
  return np.sum(np.abs(gt_boxes-pred_boxes))

#assuming we got two ground truth boxes and predicted boxes
y_true = np.array([[2., 3., 22., 25.], [44., 45., 88., 89.]])
y_pred  = np.array([[2.4, 2.8, 20.7, 23.4], [38.7, 45.5, 88.1, 89.1]])

print('L1 loss is {}'.format(L1_loss(y_true,y_pred)))
print('L2 loss is {}'.format(L2_loss(y_true,y_pred)))
```

    L1 loss is 9.499999999999988
    L2 loss is 32.809999999999974

Try the above code by giving normalized coordinates. What difference does using normalized coordinates make? (Provide your answer to the markdown cell below).

**Answer:**

**Bonus:** Implement Smooth L1 and calculate the loss again. What does smooth L1 bring to the table? How is it different than L1 and L2? Provide your answers to the markdown cell below.

```python
def smooth_L1(gt_boxes, pred_boxes):
  # Implement Smooth L1 here
  pass
```

**Answer:**

## Question 7

Implement the **focal loss** and analyze the effect of its parameters on different $p_t$ values.

The *focal loss* is designed to address the issue of extreme class imbalance problem between the foreground and background examples of one-stage object detectors.

To implement the focal loss, we should begin from the standard **binary cross entropy (CE) loss:**

\begin{equation}
\text{CE}(p, y) = \left\{ \begin{array}{cc}
- \log (p) & \text{if} \hskip 4pt y = 1 \\
- \log (1 - p) & \text{otherwise}.
\end{array}\right.
\end{equation}

where $y \in \{\pm 1\}$ indicates the ground-truth class and $p \in [0, 1]$ is the estimated probability of the model for the class with label $y = 1$. In the original paper which the focal loss is introduced, $p_t$ is defined for notational convenience as follows:

\begin{equation}
p_t = \left\{ \begin{array}{cc}
p & \text{if} \hskip 4pt y = 1 \\
1 - p & \text{otherwise},
\end{array}\right.
\end{equation}

and then we can wewrite:

\begin{equation}
\text{CE}(p, y) = \text{CE}(p_t) = - \log (p_t)
\end{equation}

One more step towards focal loss, we should consider the **balanced cross entropy** loss. It is a fairly common method for the class imbalance problem that introduces a weighting factor $\alpha \in [0, 1]$ for class $1$ and $1 - \alpha$ for class $-1$.

$\alpha$ can be set to inverse class frequency or it can be an hyperparameter that is tuned by cross-validation.

$\alpha_t$ is defined analogously to how $p_t$ is defined as the $\alpha$-balanced CE loss can be written as:

\begin{equation}
\text{CE}(p_t) = -\alpha_t \log (p_t)
\end{equation}

While the *balanced cross entropy loss* balances the importance of positive/negative samples, it is not a mechanism that differentiates between easy/hard samples. **Focal loss** is proposed to reshape the loss function so that it can reduce the weight of the easy negative samples and the model can focus the training on the hard negative samples. Focal loss is defined as follows:

\begin{equation}
\text{FL}(p_t) = -(1 - p_t)^\gamma\log (p_t)
\end{equation}

where $(1 - p_t)^\gamma$ is a modulating factor and $\gamma \geq 0 $ is a tunable *focusing* parameter.

Authors of the focal loss paper experiment with values of $\gamma \in [0, 5]$.

In practice, an $\alpha$-balanced variant of the focal loss is used, thus making the final focal loss as:

\begin{equation}
\text{FL}(p_t) = -\alpha_t(1 - p_t)^\gamma\log (p_t)
\end{equation}

Write the code for the final focal loss on the cell below. We will then test the code with different $p_t$ values to see the effect of the two parameters, namely $\alpha_t$ and $\gamma$ on the classification performance.

```python
def focal_loss(p_t, a_t, gamma):
  # Code for focal loss will be written here
  pass
```

**Answer:**

## Question 8

Explain and code the PR (Precision-Recall) curve. What happens when you use $0.7$ as threshold instead of $0.5$?

You can test your code with random precision and recall value arrays.

What is mean average precision (mAP) and why do we use it to measure the performance of our object detection model?

You can read this blog post before answering these questions: https://jonathan-hui.medium.com/map-mean-average-precision-for-object-detection-45c121a31173

**Answer:**

```python
# Code for the PR curve will be written here
```

Please elaborate on precision / recall tradeoff.

Write your answer to the cell below.

**Answer:**

inzva x KUIS AI Joint Program/Week 1 - Object Detection/Week_1_Object_Detection_Assignments.ipynb


```
#@title Copyright 2020 The Cirq Developers
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
```

# QAOA: Max-Cut

    View on QuantumAI

    Run in Google Colab

    View source on GitHub

    Download notebook

In this tutorial, we implement the quantum approximate optimization algorithm (QAOA) for determining the Max-Cut of the Sycamore processor's hardware graph (with random edge weights). To do so, we will:

1. Define a random set of weights over the hardware graph.
2. Construct a QAOA circuit using Cirq.
3. Calculate the expected value of the QAOA cost function.
4. Create an outer loop optimization to minimize the cost function.
5. Compare cuts found from QAOA with random cuts.

```
try:
    import cirq
except ImportError:
    print("installing cirq...")
    !pip install --quiet cirq
    import cirq
    print("installed cirq.")
```

## 1. Defining a random set of weights over the hardware graph
In order to make the problem easily embeddable on a quantum device, we will look at the problem of Max-Cut on the same graph that the device's qubit connectivity defines, but with random valued edge weights.

```
import cirq_google
import sympy
import numpy as np
import matplotlib.pyplot as plt

working_device = cirq_google.Sycamore
print(working_device)
```

Since a circuit covering the entire Sycamore device cannot be easily simulated, a small subset of the device graph will be used instead.

```
import networkx as nx

# Set the seed to determine the problem instance.
np.random.seed(seed=11)

# Identify working qubits from the device.
device_qubits = working_device.qubits
working_qubits = sorted(device_qubits)[:12]

# Populate a networkx graph with working_qubits as nodes.
working_graph = working_device.metadata.nx_graph.subgraph(working_qubits)

# Add random weights to edges of the graph. Each weight is a 2 decimal floating point between 0 and 5.
nx.set_edge_attributes(working_graph, {e: {'weight': np.random.randint(0, 500) / 100} for e in working_graph.edges})

# Draw the working_graph on a 2d grid
pos = {q:(q.col, -q.row) for q in working_graph.nodes()}
nx.draw(working_graph, pos=pos, with_labels=True, node_size=1000)
plt.show()
```

## 2. Construct the QAOA circuit
Now that we have created a Max-Cut problem graph, it's time to generate the QAOA circuit following [Farhi et al.](https://arxiv.org/abs/1411.4028). For simplicity $p = 1$ is chosen.

```
from cirq.contrib.svg import SVGCircuit

# Symbols for the rotation angles in the QAOA circuit.
alpha = sympy.Symbol('alpha')
beta = sympy.Symbol('beta')

qaoa_circuit = cirq.Circuit(
    # Prepare uniform superposition on working_qubits == working_graph.nodes
    cirq.H.on_each(working_graph.nodes()),

    # Do ZZ operations between neighbors u, v in the graph. Here, u is a qubit,
    # v is its neighboring qubit, and w is the weight between these qubits.
    (cirq.ZZ(u, v) ** (alpha * w['weight']) for (u, v, w) in working_graph.edges(data=True)),

    # Apply X operations along all nodes of the graph. Again working_graph's
    # nodes are the working_qubits. Note here we use a moment
    # which will force all of the gates into the same line.
    cirq.Moment(cirq.X(qubit) ** beta for qubit in working_graph.nodes()),

    # All relevant things can be computed in the computational basis.
    (cirq.measure(qubit) for qubit in working_graph.nodes()),
)
SVGCircuit(qaoa_circuit)
```

## 3. Calculating the expected value of the QAOA cost Hamiltonian
Now that we have created a parameterized QAOA circuit, we need a way to calculate expectation values of the cost Hamiltonian. For Max-Cut, the cost Hamiltonian is

$$
    H_C = \frac{1}{2} \sum_{\langle i, j\rangle} w_{ij} (1 - Z_i Z_j )
$$

where $\langle i, j \rangle$ denotes neighboring qubits, $w_{ij}$ is the weight of edge $ij$, and $Z$ is the usual Pauli-$Z$ matrix. The expectation value of this cost Hamiltonian is $\langle \alpha, \beta | H_C | \alpha, \beta \rangle$ where $|\alpha, \beta\rangle$ is the quantum state prepared by our `qaoa_circuit`. This is the cost function we need to estimate.

> Pauli-$Z$ has eigenvalues $\pm 1$. If qubits $i$ and $j$ are in the same eigenspace, then $\langle Z_i Z_j \rangle = 1$ and so $\frac{1}{2} w_{ij} \langle 1 - Z_i Z_j \rangle = 0$. In the Max-Cut language, this means that edge $ij$ does not contribute to the cost. If qubits $i$ and $j$ are in the opposite eigenspace, then $\langle Z_i Z_j \rangle = -1$ and so $\frac{1}{2} w_{ij} \langle 1 - Z_i Z_j \rangle = w_{ij}$. In the Max-Cut language, this means that edge $ij$ contributes its weight $w_{ij}$ to the cost.

To estimate the cost function, we need to estimate the (weighted) sum of all $ZZ$ pairs in the graph. Since these terms are diagonal in the same basis (namely, the computational basis), they can measured simultaneously. Given a set of measurements (samples), the function below estimates the cost function.

> *Note*: We say "estimate the cost" instead of "compute the cost" since we are sampling from the circuit. This is how the cost would be evaluated when running QAOA on a real quantum processor.

```
def estimate_cost(graph, samples):
    """Estimate the cost function of the QAOA on the given graph using the
    provided computational basis bitstrings."""
    cost_value = 0.0

    # Loop over edge pairs and compute contribution.
    for u, v, w in graph.edges(data=True):
        u_samples = samples[str(u)]
        v_samples = samples[str(v)]

        # Determine if it was a +1 or -1 eigenvalue.
        u_signs = (-1)**u_samples
        v_signs = (-1)**v_samples
        term_signs = u_signs * v_signs

        # Add scaled term to total cost.
        term_val = np.mean(term_signs) * w['weight']
        cost_value += term_val

    return -cost_value
```

Now we can sample from the `qaoa_circuit` and use `estimate_expectation` to calculate the expectation value of the cost function for the circuit. Below, we use arbitrary values for $\alpha$ and $\beta$.

```
alpha_value = np.pi / 4
beta_value = np.pi / 2
sim = cirq.Simulator()

sample_results = sim.sample(
    qaoa_circuit,
    params={alpha: alpha_value, beta: beta_value},
    repetitions=20_000
)
print(f'Alpha = {round(alpha_value, 3)} Beta = {round(beta_value, 3)}')
print(f'Estimated cost: {estimate_cost(working_graph, sample_results)}')
```

## 4. Outer loop optimization
Now that we can compute the cost function, we want to find the optimal cost. There are lots of different techniques to choose optimal parameters for the `qaoa_circuit`. Since there are only two parameters here ($\alpha$ and $\beta$), we can keep things simple and sweep over incremental pairings using `np.linspace` and track the minimum value found along the way.

```
# Set the grid size = number of points in the interval [0, 2π).
grid_size = 5

exp_values = np.empty((grid_size, grid_size))
par_values = np.empty((grid_size, grid_size, 2))

for i, alpha_value in enumerate(np.linspace(0, 2 * np.pi, grid_size)):
    for j, beta_value in enumerate(np.linspace(0, 2 * np.pi, grid_size)):
        samples = sim.sample(
            qaoa_circuit,
            params={alpha: alpha_value, beta: beta_value},
            repetitions=20000
        )
        exp_values[i][j] = estimate_cost(working_graph, samples)
        par_values[i][j] = alpha_value, beta_value
```

We can now visualize the cost as a function of $\alpha$ and $\beta$.

```
plt.title('Heatmap of QAOA Cost Function Value')
plt.xlabel(r'$\alpha$')
plt.ylabel(r'$\beta$')
plt.imshow(exp_values)
plt.show()
```

This heatmap is coarse because we selected a small `grid_size`. To see more detail in the heatmap, one can increase the `grid_size`.

## 5. Compare cuts

We now compare the optimal cut found by QAOA to a randomly selected cut. The helper function draws the `working_graph` and colors nodes in different sets different colors. Additionally, we print out the cost function for the given cut.

```
def output_cut(S_partition):
    """Plot and output the graph cut information."""

    # Generate the colors.
    coloring = []
    for node in working_graph:
        if node in S_partition:
            coloring.append('blue')
        else:
            coloring.append('red')

    # Get the weights
    edges = working_graph.edges(data=True)
    weights = [w['weight'] for (u,v, w) in edges]

    nx.draw_circular(
        working_graph,
        node_color=coloring,
        node_size=1000,
        with_labels=True,
        width=weights)
    plt.show()
    size = nx.cut_size(working_graph, S_partition, weight='weight')
    print(f'Cut size: {size}')
```

As an example, we can test this function with all nodes in the same set, for which the cut size should be zero.

```
# Test with the empty S and all nodes placed in T.
output_cut([])
```

To get cuts using the QAOA we will first need to extract the best control parameters found during the sweep:

```
best_exp_index = np.unravel_index(np.argmax(exp_values), exp_values.shape)
best_parameters = par_values[best_exp_index]
print(f'Best control parameters: {best_parameters}')
```

Each bitstring can be seen as a candidate cut in the graph. The qubits that measured 0 correspond to that qubit being in one cut partition and a qubit that measured to 1 corresponds to that qubit being in the other cut partition. Now that we've found good parameters for the `qaoa_circuit`, we can just sample some bistrings, iterate over them and pick the one that gives the best cut:

```
# Number of candidate cuts to sample.
num_cuts = 100
candidate_cuts = sim.sample(
    qaoa_circuit,
    params={alpha: best_parameters[0], beta: best_parameters[1]},
    repetitions=num_cuts
)

# Variables to store best cut partitions and cut size.
best_qaoa_S_partition = set()
best_qaoa_T_partition = set()
best_qaoa_cut_size = -np.inf

# Analyze each candidate cut.
for i in range(num_cuts):
    candidate = candidate_cuts.iloc[i]
    one_qubits = set(candidate[candidate==1].index)
    S_partition = set()
    T_partition = set()
    for node in working_graph:
        if str(node) in one_qubits:
            # If a one was measured add node to S partition.
            S_partition.add(node)
        else:
            # Otherwise a zero was measured so add to T partition.
            T_partition.add(node)

    cut_size = nx.cut_size(
        working_graph, S_partition, T_partition, weight='weight')

    # If you found a better cut update best_qaoa_cut variables.
    if cut_size > best_qaoa_cut_size:
        best_qaoa_cut_size = cut_size
        best_qaoa_S_partition = S_partition
        best_qaoa_T_partition = T_partition
```

The QAOA is known to do just a little better than random guessing for Max-Cut on 3-regular graphs at `p=1`. You can use very similar logic to the code above, but now instead of relying on the QAOA to decide your `S_partition` and `T_partition` you can just pick then randomly:

```
import random

best_random_S_partition = set()
best_random_T_partition = set()
best_random_cut_size = -9999

# Randomly build candidate sets.
for i in range(num_cuts):
    S_partition = set()
    T_partition = set()
    for node in working_graph:
        if random.random() > 0.5:
            # If we flip heads add to S.
            S_partition.add(node)
        else:
            # Otherwise add to T.
            T_partition.add(node)

    cut_size = nx.cut_size(
        working_graph, S_partition, T_partition, weight='weight')

    # If you found a better cut update best_random_cut variables.
    if cut_size > best_random_cut_size:
        best_random_cut_size = cut_size
        best_random_S_partition = S_partition
        best_random_T_partition = T_partition
```

```
print('-----QAOA-----')
output_cut(best_qaoa_S_partition)

print('\n\n-----RANDOM-----')
output_cut(best_random_S_partition)
```

For this problem instance, one should see that $p = 1$ QAOA performs better, on average, than randomly guessing.


```python
from IPython.core.display import HTML
from IPython.display import Image

import numpy as np
import sympy as sp
from sympy import oo
from utils import symplot, symdisp, round_expr

HTML("""
<style>
.output_png {
    display: table-cell;
    text-align: center;
    vertical-align: middle;
}
</style>
""")
```

<style>
.output_png {
    display: table-cell;
    text-align: center;
    vertical-align: middle;
}
</style>

# *Circuitos Elétricos I - Semana 6*

## Elementos armazenadores de energia

Comparação entre capacitores convencionais, supercapacitores e baterias de lítio. A tabela abaixo mostra as especificações  necessários para cada dispositivo armazenar ∼1 megajoule (MJ) de energia (300 watts-hora). 1 MJ de energia irá alimentar um laptop com um consumo médio de 50 W por 6 horas. Observe na primeira coluna que uma bateria de íon de lítio pode conter 1000 vezes mais energia do que um capacitor convencional.

$$
\begin{array}{|c|c|c|c|c|c|}
\hline \text { Dispositivo } & \begin{array}{c}
\text { Energia } \\
\text { específica } \\
\text { [Wh/kg]} \\
\end{array} & \begin{array}{c}
\text { Energia } \\
\text { específica } \\
\text { [MJ/kg] }
\end{array} & \begin{array}{c}
\text { Densidade de} \\
\text { Energia } \\
\text { [MJ / L] }
\end{array} & \begin{array}{c}
\text { Volume } \\
\text { requerido para } \\
\text { armazenar 1 MJ } \\
\text { [L] }
\end{array} & \begin{array}{c}
\text { Peso } \\
\text { requerido para } \\
\text { armazenar 1 MJ } \\
\text { [kg] }
\end{array} \\
\hline \begin{array}{c}
\text { Capacitor convencional} \\
\end{array} & 0.01-0.1 & 4 \times 10^{-5}-4 \times 10^{-4} & 6 \times 10^{-5}-6 \times 10^{-4} & 17000-1700 & 25000-2500 \\
\text { Supercapacitor } & 1-10 & 0.004-0.04 & 0.006-0.06 & 166-16 & 250-25 \\
\text { Bateria de Íons de Lítio } & 100-250 & 0.36-0.9 & 1-2 & 1-0.5 & 2.8-1.1 \\
\hline
\end{array}
$$

Fonte: Fawwaz Ulaby, Michel M. Maharbiz and Cynthia M. Furse, $\textit{ Circuit Analysis and Design}$, Michigan Publishing Services, 2018

## Resumo dos elementos passivos ideais de dois terminais

$$
\begin{array}{|l|c|c|c|}
\hline \text { Propriedade } & R & L & C \\
\hline \text { Relação } i-v  & i=\frac{v}{R} & i=\frac{1}{L} \int_{t_{0}}^{t} v(\tau) d \tau+i\left(t_{0}\right) & i=C \frac{d v}{d t} \\
\text { Relação } v-i & v=Ri & v=L \frac{d i}{d t} & v=\frac{1}{C} \int_{t_{0}}^{t} i(\tau) d \tau+v\left(t_{0}\right) \\
p \text { (potência }) & p=Ri^{2} & p=L i \frac{d i}{d t} & p=C v \frac{d v}{d t} \\
w \text { (energia armazenada) } & 0 & w=\frac{1}{2} L i^{2} & w=\frac{1}{2} C v^{2} \\
\text { Associação em série } & R_{\mathrm{eq}}=R_{1}+R_{2} & L_{\mathrm{eq}}=L_{1}+L_{2} & \frac{1}{C_{\mathrm{eq}}}=\frac{1}{C_{1}}+\frac{1}{C_{2}} \\
\text { Associação em paralelo } & \frac{1}{R_{\mathrm{eq}}}=\frac{1}{R_{1}}+\frac{1}{R_{2}} & \frac{1}{L_{\mathrm{eq}}}=\frac{1}{R_{1}}+\frac{1}{R_{2}} & C_{\mathrm{eq}}=C_{1}+C_{2} \\
\text { Comportamento em regime estacionário } & \text { sem mudanças } & \text { curto-circuito } & \text { circuito aberto } \\
\text { Pode } v \text { variar instantaneamente? } & \text { sim } & \text { sim } & \text { não } \\
\text { Pode } i \text { variar instantaneamente? } & \text { sim } & \text { não } & \text { sim }\\ \hline
\end{array}
$$

### Problema 1

Para o circuito abaixo, determine $v_{C1}$, $v_{C2}$ e $i_{L}$ assumindo que o circuito encontra-se em regime estacionário.

### Problema 2

No circuito abaixo, sabe-se que $i_0(t)= 50e^{-8000t}[\cos(6000t)+2\mathrm{sen}(6000t)]$ mA, para $t\geq 0^+$. Determine $v_{C}(0^+)$, $v_{L}(0^+)$ e $v_{R}(0^+)$.

```python
# define variável tempo
t = sp.symbols('t', real=True)

# expressão para a corrente no circuito
i0 = 50*sp.exp(-8000*t)*(sp.cos(6000*t)+2*sp.sin(6000*t))*1e-3

# plota gráfico da corrente
tmax = 1e-3
intervalo  = np.linspace(0, tmax, num = 1000)
symplot(t, i0, intervalo, funLabel='$i_0(t)$')
```

```python
# valores dos parâmetros do circuito
R = 320
L = 20e-3
C = 0.5e-6
```

```python
# calcula tensão no indutor
vL = L*sp.diff(i0, t)
vL = sp.simplify(vL)

print('Tensão no indutor:')
symdisp('v_L(t) = ', vL, 'V')
```

    Tensão no indutor:

$\displaystyle v_L(t) = \left(- 22.0 \sin{\left(6000 t \right)} + 4.0 \cos{\left(6000 t \right)}\right) e^{- 8000 t}\;V$

```python
symdisp('v_L(0^+) = ', vL.evalf(subs={t:0}), 'V')
```

$\displaystyle v_L(0^+) = 4.0\;V$

```python
# calcula tensão no resistor
vR = R*i0
vR = sp.simplify(vR)

print('Tensão no resistor:')
symdisp('v_R(t) = ', vR, 'V')
```

    Tensão no resistor:

$\displaystyle v_R(t) = \left(32.0 \sin{\left(6000 t \right)} + 16.0 \cos{\left(6000 t \right)}\right) e^{- 8000 t}\;V$

```python
symdisp('v_R(0^+) = ', vR.evalf(subs={t:0}), 'V')
```

$\displaystyle v_R(0^+) = 16.0\;V$

```python
# calcula tensão no capacitor (LKT)
vC = vR + vL
vC = sp.simplify(vC)

print('Tensão no capacitor:')
symdisp('v_C(t) = ', vC, 'V')
```

    Tensão no capacitor:

$\displaystyle v_C(t) = \left(10.0 \sin{\left(6000 t \right)} + 20.0 \cos{\left(6000 t \right)}\right) e^{- 8000 t}\;V$

```python
symdisp('v_C(0^+) = ', vC.evalf(subs={t:0}), 'V')
```

$\displaystyle v_C(0^+) = 20.0\;V$

```python
# checagem de vC(t) via integração de i0

vC = -(1/C)*sp.integrate(i0, (t, 0, t)) + 20
vC = sp.simplify(vC)

symdisp('v_C(t) = ', vC, 'V')
```

$\displaystyle v_C(t) = \left(10.0 \sin{\left(6000 t \right)} + 20.0 \cos{\left(6000 t \right)}\right) e^{- 8000 t}\;V$

```python
symplot(t, [vC, vR, vL], intervalo, funLabel=['$v_C(t)$ ','$v_R(t)$','$v_L(t)$'])
```

Jupyter notebooks/Circuitos Eletricos I - Semana 6.2.ipynb


# Hacker Rank Check List
- open https://regex101.com/
- open sample notebooks on local machine (iwaspoisoned3.ipynb - requests, TimeSeries_InterpExtrap, Practice_interp, bank_marketing)
- open GitHub repo
- open local notebook for testing -- copy standard template imports from this notebook
- Set up a timer so you can track your progress against the time limits
- Be ready to copy standard imports from this template into HackerRank

# Standard Template for HackerRank
- Note hacker rank doesn't seem to allow:
       - import of IPython to display dataframes
       - %matplotlib inline
       - import matplotlib.pyplot as plt
       - import matplotlib.dates as mdates
       - from pylab import rcParams
       - import matplotlib.pyplot as plt
       - import matplotlib.dates as mdates
       - from pylab import rcParams
       - rcParams['figure.figsize'] = 10, 6
       - plt.rc("font", size=14)

## Imports for Local Test

```python
%matplotlib inline
from IPython.core.display import display, HTML
import matplotlib.pyplot as plt
import matplotlib.dates as mdates
from pylab import rcParams
import matplotlib.pyplot as plt
import matplotlib.dates as mdates
from pylab import rcParams
rcParams['figure.figsize'] = 10, 6
plt.rc("font", size=14)
```

## Imports for HackerRank

```python
import os, sys, re
import calendar
from datetime import datetime
from dateutil.relativedelta import *
from scipy.stats import linregress

import collections
from collections import defaultdict, OrderedDict
import itertools
from dateutil import parser

import pandas as pd
pd.set_option('display.max_columns', 100)

import numpy as np
import scipy
import statsmodels
import statsmodels.api as sm
import statsmodels.formula.api as smf
import statsmodels.tsa.api as smt

import sympy
import requests
from bs4 import BeautifulSoup
from scipy.stats import mode
from scipy import interp

from sklearn import preprocessing, linear_model, metrics
from sklearn.linear_model import LogisticRegression, LinearRegression
from sklearn.model_selection import train_test_split, StratifiedKFold
from sklearn.cross_validation import cross_val_score

from sklearn.model_selection import GridSearchCV
from sklearn.multiclass import OneVsRestClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import roc_auc_score, f1_score, classification_report, roc_curve, auc
from sklearn.pipeline import Pipeline, FeatureUnion

# if __name__ == "__main__":
```

# HackerRank I/O
## Notes on HackerRank STDIn and STDOut
- https://www.hackerrank.com/challenges/solve-me-first/problem

## Reading a local CSV file
- lineterminator='\r'
- sep = ',', '\t'
- parse_dates = [0] # date column
- infer_datetime_format = True,
- date_parser = pd.to_datetime

### Set time-based index
- df_pass.set_index("month", inplace = True)

```python
# example
num1 = int(input())
num2 = int(input())
```

    1
    2

# Quickly load sample data to local machine (via copy/paste clipboard)

```python
df = pd.read_clipboard()
```

### Finding and loading local CSV's
- Date Time codes: https://docs.python.org/2/library/datetime.html
        Code	Meaning	Example
        %A	Weekday as full name.	      Wednesday
        %a	Weekday as abbrev. name:	    Wed
        %B	Month as locale’s full name.	June
        %b  Abbreviated month name:         Jun
        %d	Day of the month.	            06
        %m	Month as a number.	             6
        %Y	Four-digit year.	           2018
        %y	Two-digit year.	                18
        %H  is a 24-hour clock

- you can use parse dates when the information is in mulitple columns:
    parse_dates=[['Date', 'Time']]
    pd.to_datetime(df['Date'] + ' ' + df['Time'])

## STDIN READ when .tsv like data Data is expected
- zero line = number of samples in data set
- first line = header

```python
t=int(sys.stdin.readline())
my_header = sys.stdin.readline().split()

data = sys.stdin.read().splitlines()
data = [re.split(r'\t', l) for l in data]
df = pd.DataFrame(data, columns= my_header)
```

```python
# In case filedata needs to be located
dir_path = os.path.dirname(os.path.realpath(__file__))
print(dir_path)
cwd = os.getcwd()
print(cwd)
files = os.listdir(os.curdir)
print(files)

# In case data resides in its own directory below
os.chdir(path)

# Verify the data-file has been located
fname = ""
with open(fname, 'r') as fin:
    print(fin.read())

# Load the data into a dataframe
data = pd.read_csv(r'the_path_in_the_remote_machine/fname', sep='\t',
                   index_col = 0, header = 1, names = ['charge_time','battery_time'],
                   parse_dates = [0], infer_datetime_format = True, date_parser = pd.to_datetime,
                   )
```

## Missing Values

```python
## Use fillna to impute the missing values
df_bank['job'].fillna(df_bank['job'].value_counts().idxmax(), inplace=True)
df_bank['marital'].fillna(df_bank['marital'].value_counts().idxmax(), inplace=True)
df_bank['duration'].fillna(mode(df_bank['duration']).mode[0], inplace=True)
```

# Test Time Series for Seasonality

```python
plt.figure()
smt.seasonal_decompose(df_pass).plot()
plt.gcf().set_size_inches(10, 6)
plt.show()
```

# Interview Questions

## Ensemble Methods: used to improve algorithm performance and/or improve robustness
  - Bootstrapping: central concept: random sampling with replacement. Gives model a chance to learn the vairous biases, variances and features within the data set -- even applicable in small data sets. With the increase in processing power, it is much more possible to run these multiple models in parallel than ever before. Two methodologies are: Boosting and bagging
  - Bagging: run multiple prediction models in parallel and aggregate their output. Helps reduce variance in cases where overfitting is a concern. Aggregation can take the form of voting (classification) or averaging (regression).
  - Boosting: same but you weight the model's output. Usually, start by running models with equal weights on the first pass. The model then keeps track of which samples are the most frequently miss-classified and gives them heavier weights -- requiring more iteration to properly train. The model error rates are also kept track of and the better models are given more weight. Boosting is most likely to pick the better of the models included in the ensemble. It can also reduce the bias in an underfit model.
<br>
- **Summary:** Booth boosting and bagging are techniques to decrease variance -- this is why most Kaggle winners use this type of approach.

## Sorting Algorithms
https://brilliant.org/wiki/sorting-algorithms/

## Why Regularization?
[~] A way to reduce overfitting is penalize higher degree polynomials. This ensures that a higher degree polynomial is selected only if it reduces the error significantly compared to a simpler model, to overcome the penalty.

[~] Occam's razor (or Ockham's razor) is a principle from philosophy. Suppose there exist two explanations for an occurrence. In this case the simplier one is usually better. Another way of saying it is that the more assumptions you have to make, the more unlikely an explanation.

## Coefficients of Variation (like a k-factor)
In probability theory and statistics, the coefficient of variation (CV), also known as relative standard deviation (RSD), is a standardized measure of dispersion of a probability distribution or frequency distribution. It is often expressed as a percentage, and is defined as the ratio of the standard deviation to the mean. Also used to measure volatility of a security in finance

Advantages
The coefficient of variation is useful because the standard deviation of data must always be understood in the context of the mean of the data. In contrast, the actual value of the CV is independent of the unit in which the measurement has been taken, so it is a dimensionless number. For comparison between data sets with different units or widely different means, one should use the coefficient of variation instead of the standard deviation.

Disadvantages
When the mean value is close to zero, the coefficient of variation will approach infinity and is therefore sensitive to small changes in the mean. This is often the case if the values do not originate from a ratio scale.

## Non Convex Optimizaiton Methods

non-convex optimization is at least NP-hard -- worried about multiple saddle points and local minima.  Stochastic Gradient Descent

## Recommendation Engine
https://www.analyticsvidhya.com/blog/2018/06/comprehensive-guide-recommendation-engine-python/
A recommendation engine filters the data using different algorithms and recommends the most relevant items to users. It first captures the past behavior of a customer and based on that, recommends products which the users might be likely to buy.

- We can recommend items to a user which are most popular among all the users
- We can divide the users into multiple segments based on their preferences (user features) and recommend items to them based on the segment they belong to

2.3.1 Content filtering (e.g. similarty of terms -cosine distance)
2.3.2 Collaborative filtering

```python

```


 10. Markov chains

 Andrej Košir, Lucami, FE

 Kontakt: prof. dr. Andrej Košir, andrej.kosir@lucami.fe.uni-lj.si, skype=akosir_sid

 1

## ■ Intro

- Goals:
    - Markov chain modeling
    - Markov chain state analysis
- Stochastic process on finite states, most useful and used. We limit to:
    - Discrete times
    - Discrete state space (finite or countably many states)
- Case: user behavior
- Usage
    - Finite state machine analysis: software, TC system, ...
    - Models in economy
- What are questions:
    - Distributions of visits
    - Long term behavior
    - Absorption of states

 2

## ■ Sections

10.1. Definition, basic characteristics

■ Markov chain definition $\large{*}$

■ Transition matrix, distribution of states $\large{*}$

10.2. Classification of states

■ Communication of states and period $\large{*}$

■ Recurrent and transient state $\large{*}$

■ Ergodic Markov chain and return times

■ Stationary (limit) distribution $\large{*}$

■ Finite Markov chain $\large{*}$

10.3. Transition matrix estimation

■ Transition probability estimation

■ Confidence intervals and quantiles

■ Sample size determination

■ Numerical aspect

■ Case: telecommunication service user modeling

 3

## 10.1. Definition, basic characteristics

■ Markov chain definition

■ Transition matrix, distribution of states

 4

## ■ Markov chain definition

- State space $S$, $m=|S|$
    - Finite, countable, continuous

- Time $T$
    - Discrete, continuous

- Stochastic process:
$$ X_n : T \to S $$

- Discrete Markov chain is a discrete time discrete space Markov chain having Markov property
$$ P[X_{n+1}|_{X_n, X_{n-1}, \ldots, X_0}] = P[X_{n+1}|_{X_n}] = P[X_1|_{X_0}] $$

- Transition probability of one step
$$ p_{ij} = P[X_{n+1}=j|_{X_n=i}] $$

- Finite state space means there is a matrix  of transition probabilities
$$ P = [p_{ij}], \; i,j\in S. $$
    - Row sums are equal to 1.

 5

```python
import numpy as np
from sympy import Matrix
from sympy.functions import re

# Case 1
# Transition matrix
P = np.array([[0.7, 0.2, 0.1, 0.0],
              [0.2, 0.5, 0.2, 0.1],
              [0.05, 0.15, 0.6, 0.2],
              [0.0, 0.2, 0.0, 0.8]])
```

## ■ Transition matrix, distribution of states

- Transition matrix for $n$ steps
    - $p_{ij}^{(n)} = P[X_{k+n}|_{X_k=i}]$, $P^{(n)}=[p_{ij}^{(n)}]$
    - From total probability formula
    $$ P^{(n)} = P^n $$

- Distribution of states:
$$ \pi_i^n = P[X_n=i], \pi^n = [\pi_1^n, \ldots, \pi_i^n] $$
    - Initial distribution: $\pi^0$. If initial state $X_0=i_0$ is known
    $$ \pi^0=\delta_{i_0 i} $$
    - From total probability formula:
    $$ \pi^n = P^{(n)} \pi^0 $$
    - More on limit distribution later

 6

## 10.2. Classification of states

■ Communication of states and period

■ Recurrent and transient state

■ Ergodic Markov chain and return times

■ Stationary (limit) distribution

■ Finite Markov chain

 7

## ■ Communication of states and period

- Accessible state
    - State $j$ is accessible from state $i$ if $\exists n: p_{ij}^{(n)}>0$. Notation $i\leftrightarrow j$
    - Two states communicate if $i\rightarrow$ and $i \leftarrow j$, denoted by $i\leftrightarrow j$
    - „Communicate“ is equivalent relation

- Markov chain is irreducible if there are only one single communicating class

- Period of state $i$ ia denoted by $d(i)$ is greatest common divisor of all $n$ such that $P_{ii}^{(n)} > 0$:
    - If $i\leftrightarrow j$ then $d(i) = d(j)$;
    - State is aperiodic if $d(i)=1$;

- State $i$ is zero state if
$$ \lim_{n\to\infty} P_{ii}^{(n)} = 0. $$

 8

```python
# ---------------------------------------------------------------------------------------------------
# Classification of states
print ('Transition matrix P=\n', P)
print ('\nTransition matrix P^2=\n', np.linalg.matrix_power(P, 2))

# All entries of P^2 are non-zero
# => All atates are connected
```

    Transition matrix P=
     [[0.7  0.2  0.1  0.  ]
     [0.2  0.5  0.2  0.1 ]
     [0.05 0.15 0.6  0.2 ]
     [0.   0.2  0.   0.8 ]]

    Transition matrix P^2=
     [[0.535 0.255 0.17  0.04 ]
     [0.25  0.34  0.24  0.17 ]
     [0.095 0.215 0.395 0.295]
     [0.04  0.26  0.04  0.66 ]]

```python
# ---------------------------------------------------------------------------------------------------
# Decompose transition matrix
# Use Jordan decomposition
#mP = Matrix(P)
#mB, mJ = mP.jordan_form()
#B, J = np.array(mB), np.array(mJ)
B = np.array([[-0.5,-0.776313,0.659526,-0.413722],
              [-0.5,-0.212847,-0.11514,0.830469],
              [-0.5,0.283275,-0.73692,-0.123153],
              [-0.5,0.521335,0.0933625,-0.352121]])
J = np.array([[1.,0,0,0],
              [0,0.718345,0,0],
              [0,0,0.553349,0],
              [0,0,0,0.328305]])

# Test decomposition
err_mat = P - np.dot(B, np.dot(J, np.linalg.inv(B)))
print ('\nDecomposition error norm = ', np.linalg.norm(err_mat))
```

    Decomposition error norm =  7.249920892047682e-07

## ■ Recurrent and transient state

- Probability of return after exactly $n$ steps
$$ f_{ij}^n = P[X_n=j|_{X_{n-1}\ne j, \ldots, X_1\ne j, X_0=i}] $$

- Random variable of transitions from $i$ to $j$ is denoted by $T_{i}$ and its distribtion is
$$ T_{ij} \sim \left(\matrix{0 & 1 & 2 & \ldots \cr f_{ij}^0 & f_{ij}^1 & f_{ij}^2 & \ldots}\right) $$
    - Denote $T_i = T_{ii}$
    - Denote $\sum_{n=0}^\infty f_{ii}^n = f_{ii}^*$. Value
    $$ 1 - f_{ii}^* $$
    is a probability that Markov Chain newer returns to state 𝑖 after leaving it.

- State $i$ is **recurrent** if $f_{ii}^* = 1$. State is **transient** if it is not recurrent.
    - Recurrence and transience are communicating class properties

- We have: state $i$ is recurrent if and only if
$$ \sum_{n=1}^\infty P_{ii}^{(n)} = \infty $$

- Also: if $i\leftrightarrow j$ then both states are either transient or recurrent

 9

```python
# Recurent, transient?

# Eigenvalues
eigVals, eigVecs = np.linalg.eig(P.T)

# Get it sorted
idx = eigVals.argsort()[::-1]
eigVa = eigVals[idx]
eigVc = eigVecs[:,idx]

las_bf =  (1./(1-eigVa[1:4]))-1
print ('las_bf', las_bf)

sumJn = np.diag(np.insert(las_bf, 0, np.inf))
print ('\nSum J^n\n', sumJn)

sumPn = np.dot(B, np.dot(sumJn, np.linalg.inv(B)))
print ('\nSum P^n\n', sumPn)

# Since all diagonal elements of sum P^n are infinity
#  => all states are recurrent
```

    las_bf [2.55044912 1.23888596 0.48877142]

    Sum J^n
    [[       inf 0.         0.         0.        ]
     [0.         2.55044912 0.         0.        ]
     [0.         0.         1.23888596 0.        ]
     [0.         0.         0.         0.48877142]]

    Sum P^n
    [[inf inf inf inf]
     [inf inf inf inf]
     [inf inf inf inf]
     [inf inf inf inf]]

```python
def inf_sum(a_lst):
    return [(la/(1.0-la) if la < 1 else np.infty) for la in a_lst]

# ---------------------------------------------------------------------------------------------------
# Are states recurrent?
# Compute sum of P^n
sumJ = np.diag(inf_sum(np.diag(J)))
sumP = np.dot(B, np.dot(sumJ, np.linalg.inv(B)))

print ('\nSum of P^n = \n', sumP)
# => All states are reccurent
```

    Sum of P^n =
    [[inf inf inf inf]
     [inf inf inf inf]
     [inf inf inf inf]
     [inf inf inf inf]]

## ■ Ergodic Markov chain and return times

- Markov chain is **Ergodic** if it is
    - Irreducible
    - Recurrent (all states are recurrent)
    - Aperiodic (all states are aperiodic)
    - Nonzero (all states are positive)

- Expected return time:
$$ E(T_i) = \sum_{n=0}^\infty n f_{ii}^{(n)} $$

- In ergodic Markov chain
$$ \lim_{n\to\infty} P_{ii}^{(n)} = \frac{1}{E(T_i)} $$
$$ \lim_{n\to\infty} P_{ij}^{(n)} = \lim_{n\to\infty} P_{ii}^{(n)} $$

- Recurrent **zero** state $i$:
$$ \lim_{n\to\infty} P_{ii}^{(n)} = 0 \; \Leftrightarrow \;  E(T_i) = \infty  $$

- Recurrent **positive** = **ergodic** state $i$:
$$ \lim_{n\to\infty} P_{ii}^{(n)} > 0 \; \Leftrightarrow \; E(T_i) < \infty  $$

 10

```python
def inf_lim(a_lst):
    return [(0 if la < 1 else 1) for la in a_lst]

# ---------------------------------------------------------------------------------------------------
# Zero or positive states?
# Compute lim of P^n
limJ = np.diag(inf_lim(np.diag(J)))
limP = np.dot(B, np.dot(limJ, np.linalg.inv(B)))

print ('\nlim of P^n = \n', limP)

# Since all lim P^n are positive
# => all states are positive

# Expected times of visits
ETi = 1.0/np.diag(limP)

print ('\nExpected number of visits py states:\n', ETi)
```

    lim of P^n =
    [[0.21301781 0.27218919 0.18934915 0.32544385]
     [0.21301781 0.27218919 0.18934915 0.32544385]
     [0.21301781 0.27218919 0.18934915 0.32544385]
     [0.21301781 0.27218919 0.18934915 0.32544385]]

    Expected number of visits py states:
    [4.69444311 3.67391519 5.28124903 3.07272668]

## ■ Stationary (limit) distribution

- State distribution after $n$ steps
$$ \pi_i^n = P[X_n=i], \qquad \pi^n = [\pi_1^n, \ldots, \pi_i^n] $$

- Stationary (limit) distribution:
  $$ \pi = \lim_{n\to\infty} \pi^n $$
    - When uniquely defined: when all states are recurrent positive

- Ergodic MC: stationary distribution exists
- Stationary distribution of ergodic MC chain is left eigenvector of transition matrix:
  $$ \pi^\top = \pi^\top P. $$

- Note: in an ergodic MC a stationary distribution independent of initial distribution $\pi^0$.

 11

```python
# ---------------------------------------------------------------------------------------------------
# Limit distribuion
eigVals, eigVecs = np.linalg.eig(P.T)
pi = eigVecs[:, 0]/sum(eigVecs[:, 0])

print ('\neigVals=\n', eigVals)
#print '\neigVecs=\n', eigVecs
print ('\nLimit distribution =', pi)
```

    eigVals=
    [1.         0.32830522 0.71834549 0.55334929]

    Limit distribution = [0.21301775 0.27218935 0.18934911 0.32544379]

## ■ Probability of absorption

- Absorption: chain never leave a subset of states $S_0 \subset S$

- What is the probability of absorption?

- If a chain starts in transient state, then either:
    1. It leaves a set of transient states with probability of 1
    2. It enters a set of recurrent states with probability of 1

- Absorbing states $i$: the only state of an absorbing state
We have: $i$ absorbing if and only if
$P_{ii}=1$

- Let $i\in S$ be a transient state, $S_e$ a set of recurrent states:

Probability of absorption: $\pi_i(S_e)$ is the probability MC with $X_0=i$ gets absorbed into the set of states $S_e$

- We have
$$ \lim_{n\to\infty} P_{ij}^{(n)} = \pi_i(S_e)\pi_j. $$

 12

## ■ Finite Markov chain

- Finite number of states $|S|=m < \infty$.

- Each finite MC has recurrent states!

- All recurrent states are positive, that is ergodic and has finite expected return time
$$ \lim_{n\to\infty} P_{ii}^{(n)} > 0. $$

- Classification of states: all states are $S=S_m\cup S_e$, where
    - Transient states: $S_m$
    - Ergodic states: $S_e$

- Transient matrix is of a block form:
    - Ergodic states: $1, \ldots, m-K$
    - Transient state: $m-K+1, \ldots, m$
    $$ P = \left[\matrix{S & 0 \cr R & Q}\right] $$

- Fundamental matrix:
$$ N = (I - Q)^{-1}.  $$

 13

```python
# Case 2
c2P = np.array(
    [[0.7, 0.3, 0.0, 0.0],
     [0.2, 0.8, 0.0, 0.0],
     [0.05, 0.15, 0.6, 0.2],
     [0.0, 0.2, 0.0, 0.8]])

print ('Transition matrix P=\n', c2P)

```

    Transition matrix P=
     [[0.7  0.3  0.   0.  ]
     [0.2  0.8  0.   0.  ]
     [0.05 0.15 0.6  0.2 ]
     [0.   0.2  0.   0.8 ]]

```python
# ---------------------------------------------------------------------------------------------------
# Classification of states
print ('Transition matrix P=\n', c2P)
print ('\nTransition matrix P^2=\n', np.linalg.matrix_power(c2P, 2))
print ('\nTransition matrix P^3=\n', np.linalg.matrix_power(c2P, 3))

# Matrix multiplications by blocks
# => 2 x 2 block of zeros stayes the same
# => There are two connected classes of states
```

    Transition matrix P=
     [[0.7  0.3  0.   0.  ]
     [0.2  0.8  0.   0.  ]
     [0.05 0.15 0.6  0.2 ]
     [0.   0.2  0.   0.8 ]]

    Transition matrix P^2=
     [[0.55  0.45  0.    0.   ]
     [0.3   0.7   0.    0.   ]
     [0.095 0.265 0.36  0.28 ]
     [0.04  0.32  0.    0.64 ]]

    Transition matrix P^3=
     [[0.475  0.525  0.     0.    ]
     [0.35   0.65   0.     0.    ]
     [0.1375 0.3505 0.216  0.296 ]
     [0.092  0.396  0.     0.512 ]]

```python
# ---------------------------------------------------------------------------------------------------
# Limit distribuion
eigVals, eigVecs = np.linalg.eig(c2P.T)

# Get it sorted
idx = eigVals.argsort()[::-1]
eigenValues = eigVals[idx]
eigenVectors = eigVecs[:,idx]

# Normalize eigenvector
pi = eigenVectors[:, 0]/sum(eigenVectors[:, 0])

print ('\neigVals=\n', eigenValues)
#print '\neigVecs=\n', eigenVectors
print ('\nLimit distribution =', pi)
```

    eigVals=
     [1.  0.8 0.6 0.5]

    Limit distribution = [ 0.4  0.6 -0.  -0. ]

```python
# ---------------------------------------------------------------------------------------------------
# Decompose transition matrix
# Use Jordan decomposition
#mP = Matrix(c2P)
#mB, mJ = mP.jordan_form()
#B, J = np.array(mB), np.array(mJ)
c2B = np.array([[1.0, 0, 0, 10.0/4],
              [1.0, 0, 0, -3.0/2],
              [1.0, 1.0, 1.0, -7.0/8],
              [1.0, 1.0, 0, 1.0]])
c2J = np.array([[1.0,0,0,0],
              [0,4.0/5,0,0],
              [0,0,3.0/5,0],
              [0,0,0,1.0/2]])

# Test decomposition
err_mat = c2P - np.dot(c2B, np.dot(c2J, np.linalg.inv(c2B)))
print ('\nDecomposition error norm = ', np.linalg.norm(err_mat))
```

    Decomposition error norm =  1.9675159943996247e-16

```python
def prd(a,b):
    if np.isinf(a) and b == 0:
        return 0
    if np.isinf(b) and a == 0:
        return 0
    return a*b

def myDot(A,B):
    nA1, nA2, nB2 = A.shape[0], A.shape[1], B.shape[1]
    AB = np.zeros((nA1,nB2))
    for i in range(nA1):
        for j in range(nB2):
            curr = 0
            for k in range(nA2):
                curr += prd(A[i,k], B[k,j])
            AB[i, j] = curr
    return AB

# Classification of states
# Recurent, transient?

# Eigenvalues
eigVals, eigVecs = np.linalg.eig(c2P.T)

# Get it sorted
idx = eigVals.argsort()[::-1]
eigVa = eigVals[idx]
eigVc = eigVecs[:,idx]

las_bf =  (1./(1-eigVa[1:4]))-1
sumJn = np.diag(np.insert(las_bf, 0, np.inf))

sumPn = myDot(c2B, myDot(sumJn, np.linalg.inv(c2B)))
print ('\nSum P^n\n', sumPn)

# Since diagonal elements of sum P^n i=1 and i=2 are infinity
#  => states i=1 and i=2 are recurrent
# Since diagonal elements of sum P^n i=3 and i=4 are finite
#  => states i=3 and i=4 are transient

```

    Sum P^n
     [[inf inf 0.  0. ]
     [inf inf 0.  0. ]
     [inf inf 1.5 2.5]
     [inf inf 0.  4. ]]

```python
def inf_lim(a_lst):
    return [(0 if la < 1 else 1) for la in a_lst]

# ---------------------------------------------------------------------------------------------------
# Zero or positive states?
# Compute lim of P^n
limJ = np.diag(inf_lim(np.diag(c2J)))
limP = np.dot(c2B, np.dot(limJ, np.linalg.inv(c2B)))

print ('\nlim of P^n = \n', limP)

# Since all lim P^n are positive
# => all states are positive

# Expected times of visits
ETi = 1.0/np.diag(limP)

print ('\nExpected number of visits py states:\n', ETi)

```

    lim of P^n =
     [[0.4 0.6 0.  0. ]
     [0.4 0.6 0.  0. ]
     [0.4 0.6 0.  0. ]
     [0.4 0.6 0.  0. ]]

    Expected number of visits py states:
     [2.5        1.66666667        inf        inf]

    /home/nbuser/anaconda3_501/lib/python3.6/site-packages/ipykernel/__main__.py:16: RuntimeWarning: divide by zero encountered in true_divide

```python
# ---------------------------------------------------------------------------------------------------
# Limit distribuion
eigVals, eigVecs = np.linalg.eig(c2P.T)

# Get it sorted
idx = eigVals.argsort()[::-1]
eigenValues = eigVals[idx]
eigenVectors = eigVecs[:,idx]

# Normalize eigenvector
pi = eigenVectors[:, 0]/sum(eigenVectors[:, 0])

print ('\neigVals=\n', eigenValues)
#print '\neigVecs=\n', eigenVectors
print ('\nLimit distribution =', pi)
```

    eigVals=
    [1.  0.8 0.6 0.5]

    Limit distribution = [ 0.4  0.6 -0.  -0. ]

## ■ Number of visits of a transient state

- Random variable $n_i$: number of visits of a state $i$ in an arbitrary long time

- Expected number of visits of a state $j$ if initial state is $𝑖$:
 $$ E_i[n_j] < \infty $$

- We have: if state $i$ is transient:
$$ E_i[n_j] = \sum_{k=1}^\infty P_{ij}^{(k)} $$

- Also - fundamental matrix gives expected values:
$$ \left[E_i[n_j]\right] = N $$

 14

```python
# Expected number of visits of recurent states
m = 4 # |S|
K = 2 # number of recurent states

Q = c2P[K:m, K:m]
print ('Part of transition matrix Q = \n', Q)

N = np.linalg.inv(np.eye(m-K)-Q)
print ('\nFundamental matrix N =\n', N)
```

    Part of transition matrix Q =
    [[0.6 0.2]
     [0.  0.8]]

    Fundamental matrix N =
    [[2.5 2.5]
     [0.  5. ]]

## 10.3 Transition matrix estimation

■ Transition probability estimation

■ Confidence intervals and quantiles

■ Sample size determination

■ Numerical aspect

■ Case: telecommunication service user modeling

 15

## ■ Transition probability estimation

- Transition probability
$$ p_{ij} = \frac{n_k}{n} $$
is a statistical estimation;

- Parameter estimation
    - estimate and termine
        - expected value $\overline{x}$
        - standard deviation $\sigma$
        - for a selected risk level $\alpha$ compute $z_\alpha$ (if normal distribution $z_{0.05}=1.96$)
    - Confidence interval:
$$ CI = [\overline{x} - z_\alpha\frac{\sigma}{\sqrt{n}}, \overline{x} + z_\alpha\frac{\sigma}{\sqrt{n}}] $$
    - Sample size: determined from a preset confidence interval length
    $$ |CI| = 2 z_\alpha\frac{\sigma}{\sqrt{n}} $$

 16

## ■ Confidence intervals and quantiles

- Confidence interval visualization

 17

## ■ Transition matrix estimation sample size determination

- From confidence interval size

- Confidence interval for risk level $\alpha=0.05$ ($z_\alpha=1.96$)
$$ CI = [p_0 - z_\alpha\sqrt{p_0(1-p_0)/n}, p_0 + z_\alpha\sqrt{p_0(1-p_0)/n}] $$

For $p=0.65$ and $|CI|=\Delta p$ at $\alpha=0.05$
$$ n\geq \frac{4 z_\alpha^2}{\Delta p^2} p_0 (1-p_0) $$
we get
 - for $\Delta p=0.01:$ $n\geq 34959$,
 - for $\Delta p=0.02:$ $n\geq 175$,
 - for $\Delta p=0.04:$ $n\geq 88$.

 18

## ■ Numerical aspect of transition matrix computation

- We need matrix power
$$ P^n = P^{(n)} $$

- Jordan decomposition
$$ P = B J B^{-1},  $$
where $J$ is Jordan canonical form.

- For each class of states we have
$$ J = \lambda I + R $$
and
$$ P^n = B J^n B^{-1} $$

 19

## ■ Case: telecommunication service user modeling

- A set of telecommunication users $U$

- We split them into classes: $U=U_1 \cup U_2 \cup U_3 \cup U_4$
    - Very satisfied: $U_1$
    - Satisfied: $U_2$
    - Not satisfied: $U_3$
    - Churners: $U_4$

- Initial distribution: estimation of users

- Results:
    - Recurrent - transients states
    - Stationary distribution

 20

## ■ Conclusion

- Most useful: finite Markov chain
- Check the transition matrix assumption: stationarity

 21