# Chapter 2 Exercises
In this notebook we will go through the exercises of chapter 2 of Introduction to Stochastic Processes with R by Robert Dobrow.

```python
import numpy as np
```

## 2.1
A Markov chain has transition Matrix
$$
p=\left(\begin{array}{cc}
0.1 & 0.3&0.6\\
0 & 0.4& 0.6 \\
0.3 & 0.2 &0.5
\end{array}\right)
$$
With initial distribution $\alpha =(0.2,0.3,0.5)$. Find the following:

a) $P(X_7=3|X_6=2)$

b) $P(X_9=2|X_1=2,X_5=1,X_7=3)$

c) $P(X_0=3|X_1=1)$

d) $E[X_2]$

```python
a=np.array([0.2,0.3,0.5])
p = np.matrix([[0.1,0.3,0.6],[0,0.4,0.6],[0.3,0.2,0.5]])
p[1,2]
```

    0.6

```python
(p*p)[2,1]
```

    0.27

```python
a[2]*p[2,0]/(a[0]*p[0,0]+a[1]*p[1,0]+a[2]*p[2,0])
```

    0.8823529411764707

```python
e=0
pr = a*p*p
for i in range(pr.shape[1]):
    e += (i+1)*pr[0,i]
e
```

    2.3630000000000004

## 2.1
A Markov chain has transition Matrix
$$
p=\left(\begin{array}{cc}
0 & 1/2&1/2\\
1 & 0& 0 \\
1/3 & 1/3 &1/3
\end{array}\right)
$$
With initial distribution $\alpha =(1/2,0,1/2)$. Find the following:

a) $P(X_2=1|X_1=3)$

b) $P(X_1=3,X_2=1)$

c) $P(X_1=3|X_2=1)$

d) $P(X_9=1|X_1=3, X_4=1, X_7=2)$

```python
a=np.array([1/2,0,1/2])
p = np.matrix([[0,1/2,1/2],[1,0,0],[1/3,1/3,1/3]])
p[2,0]
```

    0.3333333333333333

```python
(a*p)[0,2]*p[2,0]
```

    0.13888888888888887

```python
(a*p)[0,2]*p[2,0]/((a*p)[0,0]*p[0,0]+(a*p)[0,1]*p[1,0]+(a*p)[0,2]*p[2,0])
```

    0.25

```python
(p*p)[1,0]
```

    0.0

## 2.3
Consider the Wright-Fisher model with population k=3. If the initial population has one A allele, what is the probability that there are no alleles at time 3.

```python
def calculate_combinations(n, r):
    from math import factorial
    return factorial(n) // factorial(r) // factorial(n-r)
mat = []
k = 3
for j in range(k+1):
    vec = []
    for i in range(k+1):
        vec.append(calculate_combinations(k, i)*(j/k)**i*(1-j/k)**(k-i))
    mat.append(vec)
p=np.matrix(mat)
(p**3)[1,0]
```

    0.5166895290352083

## 2.4
For the Generalñ two-state chain with transformation matrix
$$\boldsymbol{P}=\left(\begin{array}{cc}
1-p & p\\
q & 1-q
\end{array}\right)$$

And initial distribution $\alpha=(\alpha_1,\alpha_2)$, find the following:

(a) the two step transition matrix

(b) the distribution of $X_1$

### Answer
(a)

For this case the result comes from doing the matrix multiplication once i.e. finding $\boldsymbol{P}*\boldsymbol{P}$ that is:
$$\boldsymbol{P}*\boldsymbol{P}=\left(\begin{array}{cc}
(1-p)^2+pq & (1-p)p+(1-q)p\\
(1-p)q+(1-q)q & (1-q)^2+pq
\end{array}\right)$$

(b)
in this case we need to take into account alpha which would be $X_0$ and then multiply with the transition matrix, then:

$\boldsymbol{\alpha}*\boldsymbol{P}$ that is:
$$\boldsymbol{\alpha}*\boldsymbol{P}=(
(1-p)\alpha_1+q\alpha_2 , p\alpha_1+(1-q)\alpha_2)$$

## 2.5
Consider a random walk on $\{0,...,k\}$, which moves left and right with respective probabilities q and p. If the walk is at 0 it transitions to 1 on the next step. If the walk is at k it transitions to k-1 on the next step. This is calledrandom walk with reflecting bounderies. Assume that $k=3,q=1/4, p =3/4$ and the initial distribution is uniform. For the following, use technology if needed.

(a) Exhibit the transition Matrix

(b) Find $P(X_7=1|X_0=3,X_2=2,X_4=2)$

(c) Find $P(X_3=1,X_5=3)$

### Answer
(a) $$\boldsymbol{Pr}=\left(\begin{array}{cc}
0 & 1 & 0 & 0 \\
1/4 & 0 & 3/4 & 0\\
0 & 1/4 & 0 & 3/4 \\
0 & 0 & 1 & 0
\end{array}\right)$$

(b) since it is a Markov Chain, then $P(X_7=1|X_0=3,X_2=2,X_4=2)=P(X_7=1|X_4=2)=\boldsymbol{Pr}^3_{2,1}$

```python
p = np.matrix([[0,1,0,0],[1/4,0,3/4,0],[0,1/4,0,3/4],[0,0,1,0]])
(p**3)[2,1]
```

    0.296875

(c) $P(X_3=1,X_5=3)=(\alpha \boldsymbol{Pr}^3)_{1}*\boldsymbol{Pr}^2_{1,3}$

```python
a=np.matrix([1/4,1/4,1/4,1/4])
(a*p**3)[0,1]*(p**2)[1,3]
```

    0.103271484375

## 2.6
A tetrahedron die has four faces labeled 1,2,3 and 4. In repeated independent rolls of the die $R_0, R_1,...,$ let $X_n=max\{R_0,...,R_n\}$ be the maximum value after $n+1$ rolls, for $n\geq0$

(a) Give an intuitive argument for why $X_0, X_1, ... $ is a Markov Chain and exhibit its transition Matrix

(b) find P(X_3 \geq 3)

### Answer
(a) This is a Markov Chain since the value of the maximum in all the rolls does only depend on the last value given of the throw this is because the max value should not changhe based on past rolls of this given event, then $P(X_n=i|X_{n-1}=j,...)=P(X_n=i|X_{n-1}=j)$
and the transition matrix is:
$$\boldsymbol{Pr}=\left(\begin{array}{cc}
1/4 & 1/4 & 1/4 & 1/4 \\
0 & 1/2 & 1/4 & 1/4 \\
0 & 0 & 3/4 & 1/4 \\
0 & 0 & 0 & 1
\end{array}\right)$$

(b) We know that the tetrahedron has unoiform probability to get the initial value, then $\alpha=(1/4,1/4,1/4,1/4)$
then we want $\alpha*\boldsymbol{Pr}^3$

```python
p = np.matrix([[1/4,1/4,1/4,1/4],[0,1/2,1/4,1/4],[0,0,3/4,1/4],[0,0,0,1]])
a=np.matrix([1/4,1/4,1/4,1/4])
(a*p**3)[0,2:].sum()
```

    0.9375

## 2.7
Let $X_0, X_1,...$ be a Markov chain with transition matrix $P$. Let $Y_n=X_{3n}$, for $n = 0,1,2,...$ Show that $Y_0,Y_1,...$ is a Markov chain and exhibit its transition Matrix.

### Answer
Let's unravel $P(Y_k|Y_{k-1},...,Y_0)$

$$P(Y_k|Y_{k-1},...,Y_0)=P(X_{3k}|X_{3k-3},...,X_0)$$

But since $X_0, X_1,...$ is a Markov Chain, then $P(X_{3k}|X_{3k-3},...,X_0) = P(X_{3k}|X_{3k-3}) = P(Y_k|Y_{k-1})$
This means that $Y_0,Y_1,...$ is also a Markov chain.

and then the transition matrix for Y is $P_Y=P_X^3$

## 2.8
Give the Markov transition matrix for a random walk on the weighted graph in the next figure:

### Answer
$$\boldsymbol{Pr}=\left(\begin{array}{cc}
0 & 1/3 & 0 & 0 & 2/3 \\
1/10 & 1/5 & 1/5 & 1/10 & 2/5 \\
1/2 & 1/3 & 0 & 1/6 & 0\\
0 & 1/2 & 1/2 & 0 & 0\\
1/3 & 2/3 & 0 & 0 & 0
\end{array}\right)$$

## 2.8
Give the Markov transition matrix for a random walk on the weighted graph in the next figure:

### Answer
$$\boldsymbol{Pr}=\left(\begin{array}{cc}
0 & 0 & 3/5 & 0 & 2/5 \\
1/7 & 2/7 & 0 & 0 & 4/7 \\
0 & 2/9 & 2/3 & 1/9 & 0\\
0 & 1 & 0 & 0 & 0\\
3/4 & 0 & 0 & 1/4 & 0
\end{array}\right)$$

## 2.10
Consider a Markov Chain with transition Matrix

$$\boldsymbol{Pr}=\left(\begin{array}{cc}
0 & 3/5 & 1/5 & 2/5 \\
3/4 & 0 & 1/4 & 0 \\
1/4 & 1/4 & 1/4 & 1/4\\
1/4 & 0 & 1/4 & 1/2
\end{array}\right)$$
(a) Exhibit the directed, weighted transition graph for the chain

(b) The transition graph for this chain can be given as a weighted graph without directed edges. Exhibit the graph

(a)

(b)

## 2.11
You start with five dice. Roll all the dice and put aside those dice that come up 6. Then roll the remaining dice, putting aside thise dice that come up 6. and so on. Let $X_n$ be the number of dice that are sixes after n rolls.

(a)  Describe the transition matrix $\boldsymbol{Pr}$ for this Markov chain

(b) Find the probability of getting all sixes by the third play.

(c) what do you expect $\boldsymbol{Pr}^{100}$ to look like? Use tech to confirm your answer **(Good that we are doing this in jupyter notebook haha)**

(a) Note that the space for $X_n$ is $0,1,2,3,4,5$

$$\boldsymbol{Pr}=\left(\begin{array}{cc}
1 & 0 & 0 & 0 & 0 & 0 \\
1/6 & 5/6 & 0 & 0 & 0 & 0\\
\frac{1^2}{6^2}{2\choose 2} & \frac{5*1}{6^2}{2\choose 1} & \frac{5^2}{6^2}{2\choose 0} & 0 & 0 & 0\\
\frac{1^3}{6^3}{3\choose 3} & \frac{5*1^2}{6^3}{3\choose 2} & \frac{5^2*1}{6^3}{3\choose 1} & \frac{5^3}{6^3}{3\choose 0} & 0 & 0\\
\frac{1^4}{6^4}{4\choose 4} & \frac{5*1^3}{6^4}{4\choose 3} & \frac{5^2*1^2}{6^4}{4\choose 2} & \frac{5^3*1}{6^4}{4\choose 1} & \frac{5^4}{6^4}{4\choose 0} & 0\\
\frac{1^5}{6^5}{5\choose 5} & \frac{5*1^4}{6^5}{5\choose 4} & \frac{5^2*1^3}{6^5}{5\choose 3} & \frac{5^3*1^2}{6^5}{5\choose 2} & \frac{5^4*1}{6^5}{5\choose 1} & \frac{5^5}{6^5}{5\choose 0}\\
\end{array}\right) = \left(\begin{array}{cc}
1 & 0 & 0 & 0 & 0 & 0 \\
\frac{1}{6} & \frac{5}{6} & 0 & 0 & 0 & 0\\
\frac{1}{36} & \frac{10}{36} & \frac{25}{36} & 0 & 0 & 0\\
\frac{1}{216} & \frac{15}{216} & \frac{75}{216} & \frac{125}{216} & 0 & 0\\
\frac{1}{1296} & \frac{20}{1296} & \frac{150}{1296} & \frac{500}{1296} & \frac{625}{1296} & 0\\
\frac{1}{7776} & \frac{25}{7776} & \frac{250}{7776} & \frac{1250}{7776} & \frac{3125}{7776} & \frac{3125}{7776}\\
\end{array}\right)$$

It is interesting that the distribution when $X_n=i$ is the components of the binomial expansion $(1/6+5/6)^i$

(b)

```python
p = np.matrix([[1, 0, 0, 0, 0, 0],
                      [1/6, 5/6, 0, 0, 0, 0], [1/36, 10/36, 25/36, 0, 0, 0],
                      [1/216, 15/216, 75/216, 125/216, 0, 0],
                      [1/1296, 20/1296, 150/1296, 500/1296, 625/1296, 0],
                      [1/7776, 25/7776, 250/7776, 1250/7776 ,3125/7776, 3125/7776]])
(p**3)[5,0]
```

    0.013272056011374654

(c) I would expect that there is basically 100 percent posibility that there are 0 dices left without regarding how many dices you started with.

```python
(p**100).round()
```

    matrix([[1., 0., 0., 0., 0., 0.],
            [1., 0., 0., 0., 0., 0.],
            [1., 0., 0., 0., 0., 0.],
            [1., 0., 0., 0., 0., 0.],
            [1., 0., 0., 0., 0., 0.],
            [1., 0., 0., 0., 0., 0.]])

## 2.12
two urns contain k balls each. At the beginning of the experiment the urn on the left countains k red balls and the one on the right contains k blue balls. At each step, pick a ball at random from each urn and excange them. Let $X_n$ be the number of blue balls left in the urn on the left (Note that $X_0=0, X_1 = 1$). Argue the process is a Markov Chain. Find the transition matrix. This model is called the Bernoulli - Laplace process.

### Answer
This process has to be a random process because the only thing that matters at step n is to know how many blue balls are in the urn on the left. This means that even if you know all the history only the last state is the one that matters to know the probability for the next step.

$$P = \left(\begin{array}{cc}
0 & 1 & 0 & ... &0 & 0 & 0 \\
\frac{1}{k} & 0 & \frac{k-1}{k} & ... & 0 & 0 & 0\\
0 & \frac{2}{k} & 0 & ... & 0 & 0 & 0\\
... & ... & ... & ... & ... & ... & ...\\
0 & 0 & 0 & ... & \frac{k-1}{k} & 0 & \frac{1}{k}\\
0 & 0 & 0 & ... & 0 & 1 & 0\\
\end{array}\right)$$

## 2.13
see the move-to-front process in Example 2.10. Here is anorher way to organize the bookshelf. when a book is returned it is put bacl on the library shelf one position forward from where it was originally. If the book at the fron of the shelf is returned it is put back at the front of the shelf. This, if the order of books is (a,b,c,d,e) and the book d is picjed, the new order is (a,b,d,c,e). This reorganization method us called the *transposition* or *move-ahead-1* scheme. Give the transition matrix for the transportation scheme for a shelf with three books.

### Answer
remember the states are $(abc, acb, bac,bca, cab, cba)$
$$P = \left(\begin{array}{cc}
0 & p_c & p_b &p_a & 0 & 0 \\
p_b & 0 & 0 & 0 & p_c & p_a\\
p_a & p_b & 0 & p_c & 0 & 0\\
0 & 0 & p_a & 0 & p_b & p_c\\
p_c & p_a & 0 & 0 & 0 & p_b\\
0 & 0 & p_c & p_b & p_a & 0
\end{array}\right)$$

## 2.14
There are k songs in Mary's music player. The player is set to *Shuffle* mode, which plays songs uniformly at random, sampling with replacement. Thus, repeats are possible. Let $X_n$ denote the number of *unique* songs that have been heard after the nth play.

(a) show that $X_0,X_1, ...$ is a Markov chain and give the transition matrix

(b) if Mary has four songs on her music player, find the probability that all songs are heard after six plays.

### Answer
(a)
imagine that you have $P(X_n|X_{n-1},...,X_0)$ note that $X_{n+1}=max\{\xi_{n+1},X_n\}$ where $\xi_{n+1}$ is the result of the n+1 and it is independent to all $X_i$, $0\leq i\leq n$, then this means that $X_0,...,X_n$ is a Markov chain with transition matrix:
$$P = \left(\begin{array}{cc}
1/k & (k-1)/k & 0 & ... &0 & 0 & 0 \\
0 & 2/k & (k-2)/k & ... & 0 & 0 & 0\\
0 & 0 & 3/k & ... & 0 & 0 & 0\\
... & ... & ... & ... & ... & ... & ...\\
0 & 0 & 0 & ... & 0 & (k-1)/k & 1/k\\
0 & 0 & 0 & ... & 0 & 0 & 1\\
\end{array}\right)$$

## 2.15
Assume that $X_0,X_1,...$ is a two state Markov chain in $\mathcal{S}=\{0,1\}$ with transition matrix:

$$P=\left(\begin{array}{cc}
1-p & p  \\
q & 1-q
\end{array}\right)$$

The present state of the chain only depends on the previous state. We can create a bivariate process that looks back two time periods by the following contruction. Let $Z_n=(X_{n-1},X_n)$, for $n\geq1$. The sequence $Z_1,Z_2,...$ is a Markov chain with state space $\mathcal{S}X\mathcal{S}={(0,0),(1,0),(0,1),(1,1)}$ Give the transition matrix of the new chain.

### Answer
$$P = \left(\begin{array}{cc}
1-p & p & 0 & 0 \\
0 & 0 & q & 1-q \\
1-p & p & 0 & 0 \\
0 & 0 & q & 1-q
\end{array}\right)$$

## 2.16
Assume that  $P$  is a stochastic matrix with equal rows. Show that $P^n=P$, for all $n\geq1$

### Answer
let's then see $P$ as:
$$P = \left(\begin{array}{cc}
p_1 & p_2 & ... & p_k \\
p_1 & p_2 & ... & p_k \\
p_1 & p_2 & ... & p_k \\
... & ... & ... & ... \\
p_1 & p_2 & ... & p_k \\
p_1 & p_2 & ... & p_k \\
\end{array}\right)$$

First let's calculate $P^2$, then

$$P^2 = \left(\begin{array}{cc}
p_1*p_1+p_2*p_1+...+p_k*p_1 & p_1*p_2+p_2*p_2+...+p_k*p_2 & ... & p_1*p_k+p_2*p_k+...+p_k*p_k \\
p_1*p_1+p_2*p_1+...+p_k*p_1 & p_1*p_2+p_2*p_2+...+p_k*p_2 & ... & p_1*p_k+p_2*p_k+...+p_k*p_k \\
p_1*p_1+p_2*p_1+...+p_k*p_1 & p_1*p_2+p_2*p_2+...+p_k*p_2 & ... & p_1*p_k+p_2*p_k+...+p_k*p_k \\
... & ... & ... & ... \\
p_1*p_1+p_2*p_1+...+p_k*p_1 & p_1*p_2+p_2*p_2+...+p_k*p_2 & ... & p_1*p_k+p_2*p_k+...+p_k*p_k \\
p_1*p_1+p_2*p_1+...+p_k*p_1 & p_1*p_2+p_2*p_2+...+p_k*p_2 & ... & p_1*p_k+p_2*p_k+...+p_k*p_k \\
\end{array}\right)=
\left(\begin{array}{cc}
p_1*(p_1+p_2+...+p_k) & ... & p_k(p_1+p_2+...+p_k) \\
p_1*(p_1+p_2+...+p_k) & ... & p_k(p_1+p_2+...+p_k) \\
p_1*(p_1+p_2+...+p_k) & ... & p_k(p_1+p_2+...+p_k) \\
... & ... & ...  \\
p_1*(p_1+p_2+...+p_k) & ... & p_k(p_1+p_2+...+p_k) \\
p_1*(p_1+p_2+...+p_k) & ... & p_k(p_1+p_2+...+p_k) \\
\end{array}\right)=
\left(\begin{array}{cc}
p_1*(1) & ... & p_k(1) \\
p_1*(1) & ... & p_k(1) \\
p_1*(1) & ... & p_k(1) \\
... & ... & ...  \\
p_1*(1) & ... & p_k(1) \\
p_1*(1) & ... & p_k(1) \\
\end{array}\right) = P$$

Then let's see what happens for $P^3$, then
$$P^3=P^2*P=P*P=P^2=P$$

Assume it is true for $P^n$ and see what happens at $P^{n+1}$
$$P^{n+1}=P^n*P=P*P=P^2=P$$
Then this means that  $P^n=P$, for all $n\geq1$

## 2.17
Let **$P$** be the stochastic matrix. Show that $\lambda = 1$ is an eigenvalue of **$P$**. What is the associated eigenvector?

## Answer
Let's first think that it is clear that the rows all sum to one, and remember that an eigenvector asociated to an eigen value looks of the form:
$$A\overline{x}=\lambda \overline{x}$$

it is easy to note that the eigenvector we are looking at is the vector $\overline{x}=(1,1,...,1)^T$ this means that the eigenvalue asociated to this is also one

we can check this by doing the multiplication. Say we have
$$A= \left(\begin{array}{cc}
p_{1,1} & p_{1,2} & ... & p_{1,k} \\
p_{2,1} & p_{2,2} & ... & p_{2,k} \\
... & ... & ... & ... \\
p_{k,1} & p_{k,2} & ... & p_{k,k} \\
\end{array}\right)$$
where A is an stochastic matrix. Then:
$$Ax= \left(\begin{array}{cc}
p_{1,1} & p_{1,2} & ... & p_{1,k} \\
p_{2,1} & p_{2,2} & ... & p_{2,k} \\
... & ... & ... & ... \\
p_{k,1} & p_{k,2} & ... & p_{k,k} \\
\end{array}\right)\left(\begin{array}{cc}
1 \\
1 \\
...  \\
1\\
\end{array}\right)=
\left(\begin{array}{cc}
p_{1,1} + p_{1,2} + ... + p_{1,k} \\
p_{2,1} + p_{2,2} + ... + p_{2,k} \\
...  \\
p_{k,1} + p_{k,2} + ... + p_{k,k} \\
\end{array}\right)=
\left(\begin{array}{cc}
1 \\
1 \\
...  \\
1\\
\end{array}\right)$$

The way to prove this is to remember that if $A$ is an square matrix, then the eigenvalues of A are the same as its transpose's. Then, let's check if 1 is an eigenvalue by doing $det(A^T-\lambda I)$ where $\lambda$ is the eigenvalue (in this case 1) and $I$ is the identity matrix, when doing this we get that:
$$A-I= \left(\begin{array}{cc}
p_{1,1} -1 & p_{1,2} & ... & p_{1,k} \\
p_{2,1} & p_{2,2}- 1 & ... & p_{2,k} \\
... & ... & ... & ... \\
p_{k,1} & p_{k,2} & ... & p_{k,k}-1 \\
\end{array}\right)$$

And since we are dealing with the transpose, then if we sum all rows to the first row (i.e. $R_1 -> R_1+R_2+...+R_k$), then all values in the first row are zero, which means that the matrix is linearly dependand and $det(A^T- 1I)=0$, which means that 1 is an eigenvalue for $A$

## 2.18
A stochastic matrix is called *doubly* stochastic if its columns sum to 1. Let $X_0,X_1,...$ be a Markov chain on $\{ 1,...,k \}$ with doubly stochastic transition matrix and initial distribution that is uniform on $\{ 1,...,k \}$. Show that the distribution of $X_n$ is uniform on $\{ 1,...,k \}$, for all $n\geq0$

### Answer
Let's remember that then the distribution at $X_n$ is $\alpha P^n$, let's see what happens at $n=1$

$n=1$
$$X_1=\alpha P = \left(\begin{array}{cc}
1/k &1/k&...& 1/k
\end{array}\right)\left(\begin{array}{cc}
p_{1,1} & p_{1,2} & ... & p_{1,k} \\
p_{2,1} & p_{2,2} & ... & p_{2,k} \\
...  \\
p_{k,1} & p_{k,2} & ... & p_{k,k} \\
\end{array}\right)=\left(\begin{array}{cc}
p_{1,1}*1/k + p_{2,1}*1/k + ... + p_{k,1}*1/k \\
p_{1,2}*1/k + p_{2,2}*1/k + ... + p_{k,2}*1/k \\
...  \\
p_{1,k}*1/k + p_{2,k}*1/k + ... + p_{k,k}*1/k \\
\end{array}\right)^T=\left(\begin{array}{cc}
(p_{1,1} + p_{2,1} + ... + p_{k,1})*1/k\\
(p_{1,2} + p_{2,2} + ... + p_{k,2})*1/k \\
...  \\
(p_{1,k} + p_{2,k} + ... + p_{k,k})*1/k \\
\end{array}\right)^T = \left(\begin{array}{cc}
(1)*1/k\\
(1)*1/k \\
...  \\
(1)*1/k \\
\end{array}\right)^T=\alpha
$$
This last step is due to the matrix being double stochastic

Now let's $n=2$
$$X_1=\alpha P^2 = (\alpha P)*P=\alpha*p=\alpha$$

Then we assume it happens for $n$, let's see then for $n+1$
$$X_{n+1}=\alpha P^{n+1} = (\alpha P^n)*P=\alpha*p=\alpha$$

## 2.19
Let **$P$** be the transition matrix of a Markov chain on $k$ states. Let **$I$** denote the $kXk$ identity matrix. consider the matrix

$$\boldsymbol{Q}=(1-p)\boldsymbol{I}+p\boldsymbol{P}\text{  , for }0<p<1$$
show that **$Q$** is a stochastic matrix. Give the probabilistic interpretation for the dynamics of a markov chain governed by the **$Q$** matrix in terms of the original Markov chain.

### Answer
Let's construct $(1-p)\boldsymbol{I}+p\boldsymbol{P}$ and we need to check for two main things. First, $0\leq p_{i,j}\leq 1$, this is clear, since $P$ already holds this property and you are just multiplying it by two numbers that sum 1 and are less than 1. Now we need to check if $\sum_j p_{i,j}=1$, then seing the matrix:
$$(1-p)\boldsymbol{I}+p\boldsymbol{P}=\left(\begin{array}{cc}
p_{1,1}*p +(1-p) & p_{1,2}*p & ... & p_{1,k}*p \\
p_{2,1}*p & p_{2,2}*p+(1-p) & ... & p_{2,k}*p \\
... & ... & ... & ... \\
p_{k,1}*p & p_{k,2}*p & ... & p_{k,k}*p+(1-p) \\
\end{array}\right)$$
we have for row i

$p_{i,1}*p + p_{1,2}*p + ...+ p_{i,i}p+(1-p)+...+ p_{1,k}*p = p*(p_{i,1} + p_{1,2} + ...+ p_{1,k})+1-p=p*(1)+1-p=1 $

then $Q$ is a stochastic matrix.

As I see it this matrix $Q$ the only value it is modifying is the probability of transitioning from state $i$ to state $i$. so this linear combination could be used when you want to give more probability to return to the same state you are at step $n$. you can notice that the probability of going back to state $i$ when you are at state $i$ is due to the fact that is the only other state that has a real value greater than zero being added to it.

## 2.20
Let $X_0,X_1,...$ be a Markov chain with transition matrix

$$P=\left(\begin{array}{cc}
0 & 1 & 0 \\
0 & 0 & 1 \\
p & 1-p & 0
\end{array}\right)$$
for $0<p<1$. Let $g$ be the function defined by:

$$
  g(x) =
  \begin{cases}
                                   0 & \text{if $x=1$} \\
                                   1 & \text{if $x=2,3$}
  \end{cases}
$$

Let $Y_n=g(X_n)$ for $n\geq0$. Show that $Y_0,Y_1,...$ is not a Markov chain

### Answer

Let's proof this by contradiction. if we assume $Y_n$ is a Markov chain, then $P(Y_n|Y_{n-1},...,Y_0)=P(Y_n|Y_{n-1})$, but let's check this next two scenarios

## 2.21
Let $P$ and $Q$ be two transition matricies on the same state space processes, both started in some initial state $i$.

In the process $\#1$ a coin is flipped, then if it lands head, the process unfolds according to $P$. If it lands tails, the process unfolds according to $Q$.

In process $\#2$, at each step a coin is flipped. If it lands heads, the next step is chosen from $P$. If it lands tails, the next step is chosen from $Q$.

Thus, in $\#1$, one coin is initially flipped, which governsthe entire evolution of the process and in $\#2$ a coin is flipped at each step to decide the next step in the process.

Decide whether either of these processes is a Markov chain, if it is exhibit its transition matrix, if not, then explain why not.

### Answer
$#1$

For example 1 we have that it is not a Markov chain and the reason for this is because in the first step the this means that the distribution of $X_0$ is different from the distribution of $X_1$ which means that we cannot have a transition matrix for all states, although it is nice to see that once the heads is chosen, then the next steps do follow a Markov chain with either $P$ or $Q$ as their matrices.

$#2$

For example 2 we have that it is a Markov chain and the reason for this is because we can define the transition Matrix $M=0.5P+0.5Q$ as the new matrix that follows this process.

```python
## This could be a class.
def ChooseNextStep(x, mat):
    tmp_x = x*mat
    return tmp_x, np.random.choice(tmp_x.shape[1],p=np.array(tmp_x).reshape(-1))

def Simulation_21(case, P=np.matrix([[0.2,0.8],[0.2,0.8]]),Q=np.matrix([[0.8,0.2],[0.8,0.2]]),
                  simulations=1000, steps = 5, p=0.5):
    '''
    This is a function for the simulation stated in 2.21 from the book, it receives the
    two matrices and the case you are talking for either one or two and calculates its state matrices for the first
    steps we assume initial distributions uniform
    '''
    rows = P.shape[0]
    cols = P.shape[1]
    initial_dis = np.matrix([1/cols]*cols)
    sim = []
    for i in range(simulations):
        res = []
        tmp_x=initial_dis
        for j in range(steps):
            if case == 1:
                if j == 0:
                    coin = np.random.random()
                    if coin < p:
                        # Follows P
                        dist = "P"
                        tmp_x, xi = ChooseNextStep(tmp_x,P)
                    else:
                        dist = "Q"
                        tmp_x, xi = ChooseNextStep(tmp_x,Q)
                else:
                    if dist=="P":
                        tmp_x, xi = ChooseNextStep(tmp_x,P)
                    else:
                        tmp_x, xi = ChooseNextStep(tmp_x,Q)
            elif case == 2:
                coin = np.random.random()
                if coin < p:
                    # Follows P
                    tmp_x, xi = ChooseNextStep(tmp_x,P)
                else:
                    tmp_x, xi = ChooseNextStep(tmp_x,Q)
            res.append(xi)
        sim.append(res)
    return np.array(sim)

def CalcMatrixForStep(sim, step=1):
    res0 = sim[sim[:,(step-1)]==0]
    res1 = sim[sim[:,(step-1)]==1]
    prob0 = np.unique(res0[:,step], return_counts=True)[1]/res0.shape[0]
    prob1 = np.unique(res1[:,step], return_counts=True)[1]/res1.shape[0]
    return np.array([prob0,prob1])

def GetTransitionMatrix(results):
    matrices = []
    for i in range(1,results.shape[1]):
        matrices.append(CalcMatrixForStep(results,i))
    return matrices
```

```python
sim = Simulation_21(1, simulations=10000)
for elem in GetTransitionMatrix(sim):
    print(elem)
```

    [[0.68071928 0.31928072]
     [0.32472472 0.67527528]]
    [[0.6699145  0.3300855 ]
     [0.31824583 0.68175417]]
    [[0.67865078 0.32134922]
     [0.32442068 0.67557932]]
    [[0.68607443 0.31392557]
     [0.32906837 0.67093163]]

```python
sim = Simulation_21(1, simulations=100000)
for elem in GetTransitionMatrix(sim):
    print(elem)
```

    [[0.68024935 0.31975065]
     [0.32288256 0.67711744]]
    [[0.68171683 0.31828317]
     [0.32311927 0.67688073]]
    [[0.68055638 0.31944362]
     [0.32225466 0.67774534]]
    [[0.68082058 0.31917942]
     [0.32163319 0.67836681]]

```python
sim = Simulation_21(2, simulations=100000)
for elem in GetTransitionMatrix(sim):
    print(elem)
```

    [[0.50320229 0.49679771]
     [0.50051641 0.49948359]]
    [[0.49590515 0.50409485]
     [0.50063234 0.49936766]]
    [[0.50008028 0.49991972]
     [0.5005182  0.4994818 ]]
    [[0.49978013 0.50021987]
     [0.49673804 0.50326196]]

```python
sim = Simulation_21(2, simulations=100000)
for elem in GetTransitionMatrix(sim):
    print(elem)
```

    [[0.49948298 0.50051702]
     [0.50138799 0.49861201]]
    [[0.5002298  0.4997702 ]
     [0.49796825 0.50203175]]
    [[0.50042076 0.49957924]
     [0.50405271 0.49594729]]
    [[0.50585378 0.49414622]
     [0.50204918 0.49795082]]

## 2.22
Show by induction:

(a) $1+3+5+...+(2n-1)=n^2$

(b) $1^2+2^2+...+n^2=n(n+1)(2n+1)/6$

(c) for all real $x>-1$, $(1+x)^n\geq 1+nx$

### Answer
$#a$

For $n=1$

$$
\begin{align}
2(1)-1 &= 1 \\&=1^2
\end{align}$$

Assume it works for $n-1$

$$
\begin{align}
1+3+...+(2n-3)  &= (n-1)^2 \\
                &=n^2 -2n+1
\end{align}$$

Then for $n$
$$
\begin{align}
1+3+...+(2n-3)+(2n-1)  &= n^2 -2n+1 +2n-1\\
                &=n^2
\end{align}$$

Therefore is proved.

$#b$

For $n=1$

$$
\begin{align}
1(1+1)(2*1+1)/6 &= 6/6 \\&=1^2
\end{align}$$

Assume it works for $n-1$

$$
\begin{align}
1^2+2^2+...+(n-1)n^2&=(n-1)(n)(2n-1)/6
\end{align}$$

Then for $n$
$$
\begin{align}
1^2+2^2+...+n^2 &=\frac{(n-1)(n)(2n-1)}{6}+n^2\\
                &=\frac{2n^-3n^2+n+6n^2}{6}\\
                &=\frac{n(2n^2+3n+1)}{6}\\
                &=\frac{n(n+1)(2n+1)}{6}
\end{align}$$

Therefore is proved.

$#c$

For this is important to notice that for $x>-1$ means that $(1+x)>0$ for all $x$. Now let's see what happens for $n=1$

$$
\begin{align}
(1+x)^1 &= (1+1*x)\\
1+x &\geq 1+x
\end{align}$$

Assume it works for $n-1$

$$
\begin{align}
(1+x)^{n-1}&\geq1+(n-1)x
\end{align}$$

Then for $n$
$$
\begin{align}
(1+x)^{n} &= (1+x)^{n-1}*(1+x)\\
          &\geq 1+(n-1)x*(1+x)\\
\\ \text{this last step because $1+x > 0$} \\ \\
1+(n-1)x*(1+x)    &=1+(n-1)x+x+(n-1)x^2\\
                  &=1+n*x+ (n-1)x^2\\
\\ \text{but since $(n-1)x^2\geq 0$ for all $x>1$}\\ \\
1+n*x+ (n-1)x^2 &\geq 1+n*x \\
\\ \text{=>} \\ \\
(1+x)^{n} &\geq 1+n*x+ (n-1)x^2 \\
          &\geq 1+n*x \\
\\ \text{by transitivity relation} \\ \\
(1+x)^{n} &\geq 1+n*x \\
\end{align}$$

Therefore is proved.

## 2.23
Simulate the first 20 letters (vowel/consonant) of the Pushkin poem Markov chain of Example 2.2.

$$P=\left(\begin{array}{cc}
0.175 & 0.825 \\
0.526 & 0.474
\end{array}\right)$$

### Answer

```python
class PushkinLetters():
    def __init__(self, P=np.matrix([[0.175,0.825],[0.526,0.474]]), init= np.array([8.638/20, 11.362/20])):
        self.P=P
        self.init = init
    def sample(self, simlist, dist):
        vowel = 0
        consonant = 1
        if (np.random.random()<dist[0]):
            simlist.append(vowel)
        else:
            simlist.append(consonant)
    def parseSimulation(self, toParse):
        return ["vowel" if x == 0 else "consonant" for x in toParse]
    def OneSimulation(self, steps = 20):
        sim = []
        self.sample(sim, self.init)
        for i in range(1,steps):
            self.sample(sim, np.array(self.P[sim[i-1]]).reshape(-1))
        return self.parseSimulation(sim)
    def Simulation(self, simulations):
        vowel = 0
        consonant = 1
        results = []
        for i in range(0,simulations):
            results.append(self.OneSimulation())
        return results
```

```python
PushkinLetters().OneSimulation()
```

    ['vowel',
     'consonant',
     'vowel',
     'consonant',
     'consonant',
     'consonant',
     'vowel',
     'consonant',
     'vowel',
     'consonant',
     'vowel',
     'consonant',
     'vowel',
     'vowel',
     'consonant',
     'consonant',
     'vowel',
     'consonant',
     'consonant',
     'consonant']

## 2.24
Simulate 50 steps of the random walk on the graph in Figure2.1. Repeat the simulation 10 times. How many of your simulations end at vertex c? Compare with the exact long-term probability the walk visits c.

With transition matrix uniform across vertex as described in the frog example, then:
$$P=\left(\begin{array}{cc}
0 & 1 & 0 & 0 & 0 & 0\\
1/4 & 0 & 1/4 & 1/4 & 1/4 & 0\\
0 & 1/4 & 0 & 1/4 & 1/4 & 1/4\\
0 & 1/4 & 1/4 & 0 & 1/4 & 1/4\\
0 & 1/3 & 1/3 & 1/3 & 0 & 0\\
0 & 0 & 1/2 & 1/2 & 0 & 0
\end{array}\right)$$

### Answer

```python
class FrogJump():
    def __init__(self, P=np.matrix([[0 , 1 , 0 , 0 , 0 , 0],
                        [1/4 , 0 , 1/4 , 1/4 , 1/4 , 0],
                        [0 , 1/4 , 0 , 1/4 , 1/4 , 1/4],
                        [0 , 1/4 , 1/4 , 0 , 1/4 , 1/4],
                        [0 , 1/3 , 1/3 , 1/3 , 0 , 0],
                        [0 , 0 , 1/2 , 1/2 , 0 , 0]]),
                 init= np.array([1/6]*6)):
        self.P=P
        self.init = init
    def sample(self, simlist, dist):
        simlist.append(np.random.choice(len(dist),p=dist))
    def OneSimulation(self, steps = 50):
        sim = []
        self.sample(sim, self.init)
        for i in range(1,steps):
            self.sample(sim, np.array(self.P[sim[i-1]]).reshape(-1))
        return sim
    def Simulation(self, simulations):
        vowel = 0
        consonant = 1
        results = []
        for i in range(0,simulations):
            results.append(self.OneSimulation())
        return results
```

```python
np.array(FrogJump().Simulation(10))[:,-1]
```

    array([3, 2, 4, 2, 3, 4, 1, 2, 3, 3])

```python
P=np.matrix([[0 , 1 , 0 , 0 , 0 , 0],
            [1/4 , 0 , 1/4 , 1/4 , 1/4 , 0],
            [0 , 1/4 , 0 , 1/4 , 1/4 , 1/4],
            [0 , 1/4 , 1/4 , 0 , 1/4 , 1/4],
            [0 , 1/3 , 1/3 , 1/3 , 0 , 0],
            [0 , 0 , 1/2 , 1/2 , 0 , 0]])
(P**30)[0,:]
```

    matrix([[0.05555595, 0.22222121, 0.22222273, 0.22222273, 0.16666667,
             0.11111072]])

## 2.25
The behavior of dolphins in the presence of tour boats in Patagonia, Argentina is studied in Dans et al. (2012). A Markov chain model is developed, with state space consisting of five primary dolphin activities (socializing, traveling, milling, feeding, and resting). The following transition matrix is
obtained.

$$P=\left(\begin{array}{cc}
0.84 & 0.11 & 0.01 & 0.04 & 0 \\
0.03 & 0.8 & 0.04 & 0.1 & 0.03 \\
0.01 & 0.15 & 0.7 & 0.07 & 0.07 \\
0.03 & 0.19 & 0.02 & 0.75 & 0.01 \\
0.03 & 0.09 & 0.05 & 0 & 0.83
\end{array}\right)$$

### Answer

```python
(np.matrix([
[0.84 , 0.11 , 0.01 , 0.04 , 0],
[0.03 , 0.8 , 0.04 , 0.1 , 0.03],
[0.01 , 0.15 , 0.7 , 0.07 , 0.07],
[0.03 , 0.19 , 0.02 , 0.75 , 0.01],
[0.03 , 0.09 , 0.05 , 0 , 0.83]
])**100)[0]
```

    matrix([[0.14783582, 0.41492537, 0.0955597 , 0.2163806 , 0.1252985 ]])

## 2.26
In computer security applications, a honeypot is a trap set on a network to detect and counteract computer hackers. Honeypot data are studied in Kimou et al. (2010) using Markov chains. The authors obtain honeypot data from a central database and observe attacks against four computer ports—80, 135, 139, and 445—over 1 year. The ports are the states of a Markov chain along with a state corresponding to no port is attacked. Weekly data are monitored, and the port most often attacked during the week is recorded. The estimated Markov transition matrix for weekly attacks is
$$P=\left(\begin{array}{cc}
0 & 0 & 0 & 0 & 1 \\
0 & 8/13 & 3/13 & 1/13 & 1/13 \\
1/16 & 3/16 & 3/8 & 1/4 & 1/8 \\
0 & 1/11 & 4/11 & 5/11 & 1/11 \\
0 & 1/8 & 1/2 & 1/8 & 1/4
\end{array}\right)$$
with initial distribution $\alpha = (0, 0, 0, 0, 1)$.

(a) Which are the least and most likely attacked ports after 2 weeks?

(b) Find the long-term distribution of attacked ports.

### Answer

```python
P=np.matrix([
[0 , 0 , 0, 0 , 1],
[0 , 8/13 , 3/13 , 1/13, 1/13 ],
[1/16 , 3/16 , 3/8 , 1/4, 1/8 ],
[0 , 1/11 , 4/11 , 5/11 , 1/11],
[0 , 1/8 , 1/2 , 1/8 , 1/4 ]])
a=np.array([0,0,0,0,1])
parser=[80,135, 139, 445, 'No attack']
```

```python
print("the most likely port to be attacked after two weeks is:",parser[(a*P**2).argmax()],"\n"+
      "the least likely port to be attacked after two weeks is:",parser[(a*P**2).argmin()])
```

    the most likely port to be attacked after two weeks is: 139
    the least likely port to be attacked after two weeks is: 80

```python
print("the long last distribution is:",(P**100)[0])
```

    the long last distribution is: [[0.02146667 0.26693333 0.34346667 0.22733333 0.1408    ]]

## 2.27
See gamblersruin.R. Simulate gambler’s ruin for a gambler with initial stake $\$2$, playing a fair game.

(a) Estimate the probability that the gambler is ruined before he wins $\$5$.

(b) Construct the transition matrix for the associated Markov chain. Estimate
the desired probability in (a) by taking high matrix powers.

(c) Compare your results with the exact probability.

### Answer

```python
class GamblingWalk():
    '''
    This class is used to simulate a random walk, it is flexible to take different
    outcomes for the results of throwing the coin and you can have different limits to when stop the experiment
    The plotting functions are handy to see the final results.
    '''
    def __init__(self, initial_money=50, prob=1/2, min_state=0, max_state=100, outcomes = [-1,1]):
        self.prob = prob
        self.init_money = initial_money
        self.walk = []
        self.min_state = min_state
        self.max_state = max_state
        self.outcomes = outcomes
    def P_w(self, sim=100):
        results = []
        for i in range(sim):
            res = self.randomWalk()
            results.append(res[1])
        return sum(results)/len(results)
    def randomWalk(self):
        money = self.init_money
        self.walk = [int(money)]
        win = False
        while True:
            money += np.random.choice(self.outcomes,1,p=[1-self.prob,self.prob])
            self.walk.append(int(money))
            if (money <= self.min_state) or (money >= self.max_state):
                win=(money >= self.max_state)
                break
        return len(self.walk), win
    def plotWalk(self):
        if(len(self.walk)==0):
            self.randomWalk()
        plt.plot(range(len### Answer(self.walk)), self.walk)
        plt.xlabel("Time")
        plt.ylabel("Money")
        plt.show()
```

```python
wlk = GamblingWalk(initial_money=2, max_state=5)
1-wlk.P_w(10000)
```

    array([0.5985])

(b) The transition Matrix is given by:

$$P=\left(\begin{array}{cc}
1 & 0 & 0 & 0 & 0 & 0 \\
1/2 & 0 & 1/2 & 0 & 0 & 0 \\
0 & 1/2 & 0 & 1/2 & 0 & 0 \\
0 & 0 & 1/2 & 0 & 1/2 & 0 \\
0 & 0 & 0 & 1/2 & 0 & 1/2 \\
0 & 0 & 0 & 0 & 0 & 1 \\
\end{array}\right)$$

```python
P=np.matrix([[1 , 0 , 0 , 0 , 0 , 0],
            [1/2 , 0 , 1/2 , 0 , 0 , 0],
            [0 , 1/2 , 0 , 1/2 , 0 , 0],
            [0 , 0 , 1/2 , 0 , 1/2 , 0],
            [0 , 0 , 0 , 1/2 , 0 , 1/2],
            [0 , 0 , 0 , 0 , 0 ,1]])
a=np.array([0,0,1,0,0,0])
(a*P**100).round(4)
```

    array([[0.6, 0. , 0. , 0. , 0. , 0.4]])

(c) As seen from the results from (a) and (b) we can see that the simulation is a bit off, even after 10,000 simulations.


# sympy

Since we want to work interactively with `sympy`, we will import the complete module. Note that this polutes the namespace, and is not recommended in general.

```python
from sympy import *
```

Enable pretty printing in this notebook.

```python
init_printing()
```

## Expression manipulation

Define a number of symbols to work with, as well as an example expression.

```python
x, y, a, b, c = symbols('x y a b c')
```

```python
expr = (a*x**2 - b*y**2 + 5)/(c*x + y)
```

Check the expression's type.

```python
expr.func
```

    sympy.core.mul.Mul

Although the expression was defined as a divisino, it is represented as a multiplicatino by `sympy`.  The `args` attribute of an expressions stores the operands of the top-level operator.

```python
expr.args
```

Although the first factor appears to be a division, it is in fact a power.  The denominator of this expression would be given by:

```python
expr.args[0].func
```

    sympy.core.power.Pow

```python
expr.args[0].args[0]
```

The expression $\frac{1}{a x + b}$ can alternatively be defined as follows, which highlights the internal representation of expressions.

```python
expr = Pow(Add(Mul(a, x), b), -1)
```

```python
pprint(expr)
```

       1
    ───────
    a⋅x + b

```python
expr.args
```

    (a*x + b, -1)

```python
expr.args[0].args[0]
```

    b

```python
expr = x**2 + 2*a*x + y**2
```

```python
expr2 = expr.subs(a, y)
expr2
```

Most expression manipulation algorithms can be called as functions, or as methods on expressions.

```python
factor(expr2)
```

```python
expr2.factor()
```

```python
x, y = symbols('x y', positive=True)
```

```python
(log(x) + log(y)).simplify()
```

## Calculus

### Series expansion

```python
x, a = symbols('x a')
```

```python
expr = sin(a*x)/x
```

```python
expr2 = series(expr, x, 0, n=7)
```

```python
expr2
```

A term of a specific order in a given variable can be selected easily.

```python
expr2.taylor_term(2, x)
```

When the order is unimportant, or when the expression should be used to define a function, the order term can be removed.

```python
expr2.removeO()
```

Adding two series deals with the order correctly.

```python
s1 = series(sin(x), x, 0, n=7)
```

```python
s2 = series(cos(x), x, 0, n=4)
```

```python
s1 + s2
```

### Derivatives and integrals

```python
expr = a*x**2 + b*x + c
```

```python
expr.diff(x)
```

```python
expr.integrate(x)
```


# Normal distribution

Given normal density and distribution functions for N(0,1)

\begin{equation}
  f_X(x)=\frac{1}{\sqrt{2\pi}}e^{-x^2/2}
\end{equation}

\begin{equation}
  F_X(x)=\int_\infty^x \frac{1}{\sqrt{2\pi}}e^{-u^2/2}du
\end{equation}

What is the probability that $4.0 < x < 4.5$?

```python
print("Probability from table: %.6f" % (0.999997 - 0.999968))
```

    Probability from table: 0.000029

```python
from numpy import pi, sqrt, exp
from scipy.integrate import quad

fx = lambda x: 1/sqrt(2*pi)*exp(-x**2/2)
prob = quad(fx, 4, 4.5)
print("Probability from integration: %.6f (%.2e)" % prob)
```

    Probability from integration: 0.000028 (3.14e-19)


### DEMSLV04

# Compute fixedpoint of $f(x, y)= [x^2 + y^3; xy - 0.5]$

Compute fixedpoint of

\begin{equation}
f(x, y)= \begin{bmatrix}x^2 + y^3 \\ xy - 0.5 \end{bmatrix}
\end{equation}

using Newton, Broyden, and function iteration methods.

Initial values generated randomly.  Some algorithms may fail to converge, depending on the initial value.

True fixedpoint is $x = -0.09$,  $y=-0.46$.

```python
from demos.setup import np, tic, toc
from compecon import NLP
np.random.seed(12)
```

### Set up the problem

```python
def g(z):
    x, y = z
    return np.array([x **2 + y ** 3, x * y - 0.5])

problem_as_fixpoint = NLP(g, maxit=1500)
```

### Equivalent Rootfinding Formulation

```python
def f(z):
    x, y = z
    fval = [x - x ** 2 - y ** 3,
            y - x * y + 0.5]
    fjac = [[1 - 2 * x, -3 * y **2],
            [-y, 1 - x]]

    return np.array(fval), np.array(fjac)

problem_as_zero = NLP(f, maxit=1500)
```

### Randomly generate starting point

```python
xinit = np.random.randn(2)
```

### Compute fixed-point using Newton method

```python
t0 = tic()
z1 = problem_as_zero.newton(xinit)
t1 = 100 * toc(t0)
n1 = problem_as_zero.fnorm
```

### Compute fixed-point using Broyden method

```python
t0 = tic()
z2 = problem_as_zero.broyden(xinit)
t2 = 100 * toc(t0)
n2 = problem_as_zero.fnorm
```

### Compute fixed-point using function iteration

```python
t0 = tic()
z3 = problem_as_fixpoint.fixpoint(xinit)
t3 = 100 * toc(t0)
n3 = np.linalg.norm(problem_as_fixpoint.fx - z3)
```

```python
print('Hundredths of seconds required to compute fixed-point of ')
print('\n\t\tg(x1,x2)=[x1^2+x2^3; x1*x2-0.5]')
print('\nusing Newton, Broyden, and function iteration methods, starting at')
print('\n\t\tx1 = {:4.2f}  x2 = {:4.2f}\n\n'.format(*xinit))
print('Method       Time   Norm of f         x1       x2\n', '-' * 48)
print('Newton   {:8.2f}    {:8.0e}     {:5.2f}     {:5.2f}'.format(t1, n1, *z1))
print('Broyden  {:8.2f}    {:8.0e}     {:5.2f}     {:5.2f}'.format(t2, n2, *z2))
print('Function {:8.2f}    {:8.0e}     {:5.2f}     {:5.2f}'.format(t3, n3, *z3))
```

    Hundredths of seconds required to compute fixed-point of

    		g(x1,x2)=[x1^2+x2^3; x1*x2-0.5]

    using Newton, Broyden, and function iteration methods, starting at

    		x1 = 0.47  x2 = -0.68

    Method       Time   Norm of f         x1       x2
     ------------------------------------------------
    Newton       0.80       2e-15     -0.09     -0.46
    Broyden      0.10       8e-10     -0.09     -0.46
    Function     0.05       8e-09     -0.09     -0.46

notebooks/slv/04 Compute fixedpoint of f(x1,x2)= [x1 2+x2 3; x1 x2 - 0.5].ipynb


# The Jones Calculus: Symbolic Algebra

**Scott Prahl**

**April 2020**

```python
#this must be run first
%matplotlib inline

import matplotlib
import numpy as np
import matplotlib.pyplot as plt

import sympy
import pypolar.sym_jones as sym_jones
sympy.init_printing(use_latex='mathjax')

```

## Background

### Jones Vector for Linearly Polarized Light

A Jones vector is a 2x1 matrix that can represent fully polarized light.  Linearly polarized light is relatively simple
$$
\mbox{horizontal linearly polarized light}=
\left[\begin{array}{c}
1\\
0\\
\end{array}\right]
$$
and

```python
sym_jones.field_horizontal()
```

$\displaystyle \left[\begin{matrix}1\\0\end{matrix}\right]$

$$
\mbox{vertical linearly polarized light}=
\left[\begin{array}{c}
0\\
1\\
\end{array}\right]
$$

```python
sym_jones.field_vertical()
```

$\displaystyle \left[\begin{matrix}0\\1\end{matrix}\right]$

$$
\mbox{linearly polarized at $\theta$ from horizontal}=
\left[\begin{array}{c}
\cos\theta\\
\sin\theta\\
\end{array}\right]
$$
So the vector for linearly polarized light at 45 degrees can be written in python as

```python
theta = sympy.Symbol('theta', real=True)
sym_jones.field_linear(theta)
```

$\displaystyle \left[\begin{matrix}\cos{\left(\theta \right)}\\\sin{\left(\theta \right)}\end{matrix}\right]$

### Jones Vectors for Circularly Polarized Light

For circularly polarized light, the direction of polarization rotates through an entire circle every wavelength, and retains a constant magnitude.

Right circularly polarized light has the direction of polarization rotating clockwise when looking into the beam.
$$
\mbox{right linearly polarized light}=
{1\over\sqrt{2}}\left[\begin{array}{c}
j\\
1\\
\end{array}\right]
$$

```python
sym_jones.field_right_circular()
```

$\displaystyle \left[\begin{matrix}\frac{\sqrt{2}}{2}\\- \frac{\sqrt{2} i}{2}\end{matrix}\right]$

$$
\mbox{vertical linearly polarized light}=
{1\over\sqrt{2}}\left[\begin{array}{c}
1\\
j\\
\end{array}\right]
$$
So the vector for left circularly polarized light can be written in python as

```python
sym_jones.field_left_circular()
```

$\displaystyle \left[\begin{matrix}\frac{\sqrt{2}}{2}\\\frac{\sqrt{2} i}{2}\end{matrix}\right]$

### Elliptically polarized light
In general, the expression for elliptically polarized light is
$$
\mathbf{I}=
\left[\begin{array}{c}
A\\
Be^{j\delta}\\
\end{array}\right]
$$

### Irradiance
The scalar intensity of the beam is the square of the field $E\cdot E^*$,
$$
\left[\begin{array}{c}
A\\
Be^{j\delta}\\
\end{array}\right]
\cdot
\left[\begin{array}{c}
A^* &
B^*e^{-j\delta}\\
\end{array}\right]
$$
or
$$
I= A A^* + B B^*
$$
where $A^*$ is the complex conjugate of $A$.

## Jones Matrix for Linear Polarizer

The Jones matrix representing a perfect horizontal polarizer is
$$
\mathbf{H}=\mbox{Horizontal Linear Polarizer}=\left[\begin{array}{cc}
1 & 0\\
0 & 0\\
\end{array}\right]
$$

The matrix representing a rotation through an angle $\theta$ is
$$
\mathbf{R}(\theta)=\mbox{Rotation by $\theta$}=\left[\begin{array}{cc}
\cos\theta & \sin \theta\\
-\sin\theta & \cos\theta\\
\end{array}\right]
$$

The matrix representing a linear polarizer oriented at an angle of $\theta$ to the horizontal axis can by rotating the beam so the axis coincides with the horizontal axis and then unrotating by the same amount
$$
\mbox{Linear Polarizer at $\theta$} = \mathbf{R}(-\theta)\cdot\mathbf{H}\cdot\mathbf{R}(\theta)
$$

```python
## By rotation
theta = sympy.Symbol('theta', real=True)
H = sym_jones.op_linear_polarizer(0)
R = sym_jones.op_rotation(theta)
linear = sym_jones.op_rotation(-theta) * H * sym_jones.op_rotation(theta)
linear
```

$\displaystyle \left[\begin{matrix}\cos^{2}{\left(\theta \right)} & \sin{\left(\theta \right)} \cos{\left(\theta \right)}\\\sin{\left(\theta \right)} \cos{\left(\theta \right)} & \sin^{2}{\left(\theta \right)}\end{matrix}\right]$

```python
## Directly
sym_jones.op_linear_polarizer(theta)
```

$\displaystyle \left[\begin{matrix}\cos^{2}{\left(\theta \right)} & \sin{\left(\theta \right)} \cos{\left(\theta \right)}\\\sin{\left(\theta \right)} \cos{\left(\theta \right)} & \sin^{2}{\left(\theta \right)}\end{matrix}\right]$

### Retarders and Wave Plates

An optical retarder has a fast and a slow axis.  Light polarized parallel to the fast axis  moves through the wave plate faster than light polarized perpendicular to the fast axis.  A phase lag (or retardance) $\phi$ is created between the two polarization states.  The Jones matrix for a retarder with its fast axis oriented at $\theta$ is
$$
\left[\begin{array}{cc}
\cos{\phi\over2} + j\sin{\phi\over2}\cos2\theta & j \sin{\phi\over2}\sin 2\theta\\
j \sin{\phi\over2}\sin 2\theta & \cos{\phi\over2} - j\sin{\phi\over2}\cos2\theta
\end{array}\right]
$$

```python
## are these the same??
phi = sympy.Symbol('phi', real=True)
sym_jones.op_retarder(theta,phi)
```

$\displaystyle \left[\begin{matrix}e^{\frac{i \phi}{2}} \cos^{2}{\left(\theta \right)} + e^{- \frac{i \phi}{2}} \sin^{2}{\left(\theta \right)} & 2 i \sin{\left(\frac{\phi}{2} \right)} \sin{\left(\theta \right)} \cos{\left(\theta \right)}\\2 i \sin{\left(\frac{\phi}{2} \right)} \sin{\left(\theta \right)} \cos{\left(\theta \right)} & e^{\frac{i \phi}{2}} \sin^{2}{\left(\theta \right)} + e^{- \frac{i \phi}{2}} \cos^{2}{\left(\theta \right)}\end{matrix}\right]$

```python
theta = sympy.Symbol('theta', real=True)

slm = sym_jones.op_retarder(sympy.pi/2,phi)
i0 = sym_jones.field_linear(sympy.pi/4)
anal = sym_jones.op_linear_polarizer(theta)
result = slm * i0
result/result[0]
```

$\displaystyle \left[\begin{matrix}1\\e^{i \phi}\end{matrix}\right]$

HaIf the thickness of the retarder is $d$ and the fast and slow indices of refraction are $n_o$ and $n_e$, then the retardance for a wavelength $\lambda$ is
$$
\phi(\lambda) = {2\pi\over\lambda} d (n_o-n_e)
$$
A half-wave plate has $\phi(\lambda)=\pi$ and a Jones matrix
$$
\mathbf{M}_\mathrm{\lambda/2}=j \left[\begin{array}{cc}
\cos2\theta & \sin 2\theta\\
\sin2\theta & -\cos2\theta
\end{array}\right]
$$

```python
sympy.simplify(sym_jones.op_half_wave_plate(theta))
```

$\displaystyle \left[\begin{matrix}i \cos{\left(2 \theta \right)} & i \sin{\left(2 \theta \right)}\\i \sin{\left(2 \theta \right)} & - i \cos{\left(2 \theta \right)}\end{matrix}\right]$

A quarter-wave plate has $\phi(\lambda)=\pi/2$ and
$$
\mathbf{M}_\mathrm{\lambda/4}={1\over\sqrt{2}}\left[\begin{array}{cc}
1 + j\cos2\theta & j\sin2\theta\\
j\sin2\theta & 1-j\cos 2\theta
\end{array}\right]
$$

```python
## same??
sym_jones.op_quarter_wave_plate(theta)
```

$\displaystyle \left[\begin{matrix}e^{- \frac{i \pi}{4}} \sin^{2}{\left(\theta \right)} + e^{\frac{i \pi}{4}} \cos^{2}{\left(\theta \right)} & \sqrt{2} i \sin{\left(\theta \right)} \cos{\left(\theta \right)}\\\sqrt{2} i \sin{\left(\theta \right)} \cos{\left(\theta \right)} & e^{\frac{i \pi}{4}} \sin^{2}{\left(\theta \right)} + e^{- \frac{i \pi}{4}} \cos^{2}{\left(\theta \right)}\end{matrix}\right]$

A retarder exhibits different retardances at a different wavelengths.  Because $n_o-n_e$ changes slowly with wavelength, the phase at $\lambda_1$ of a wave plated designed for $\lambda_0$ is
$$
\phi(\lambda_1) = \phi(\lambda_0) {\lambda_0\over\lambda_1}
$$
This is relevant because you will be using phase plates that are created for wavelengths other than 633nm.

### Optical Isolator

Linearly polarized light passes through the polarizer and is converted to right hand circularly polarized light by the quarter wave plate. Upon reflection, this beam will change handedness, and upon passage back through the quarter wave plate it is linear again but orthogonal to the polarizer.

## Malus's Law

Show that the transmission of linearly-polarized light as it passes through a linear analyzer at an angle $\theta$.

```python
incident = sym_jones.field_horizontal()
linear = sym_jones.op_linear_polarizer(theta)
result = linear * incident
intensity = sym_jones.intensity(result)
fraction = sympy.simplify(intensity)
fraction
```

$\displaystyle \cos^{2}{\left(\theta \right)}$

```python
theta = np.linspace(0, 180, 100)
plt.plot(theta, np.cos(np.radians(theta))**2)

plt.title('Malus\'s Law')
plt.xlabel( "Analyzer Angle (degrees)" )
plt.ylabel( "Transmission" )
plt.show()
```

## Crossed polarizers with quarter-wave plate between

Show that the total transmission of horizontally-polarized light that passes through a quarter-wave plate oriented at an angle $\theta$ and then through a vertical analyzer is $2\cos^2\theta\sin^2\theta$ and plot the transmitted light as the polarizer is rotated through 180 degrees.

```python
theta = sympy.Symbol('theta', real=True)
incident = sym_jones.field_horizontal()

plate = sym_jones.op_quarter_wave_plate(theta)
analyzer = sym_jones.op_linear_polarizer(sympy.pi/2)
result = analyzer * plate * incident
fraction = sym_jones.intensity(result)
fraction

```

$\displaystyle 2 \sin^{2}{\left(\theta \right)} \cos^{2}{\left(\theta \right)}$

```python
theta = np.linspace(0, 180, 180)
th = 2*np.radians(theta)
plt.plot(theta, 0.5*np.sin(th)**2)

plt.title('Horizontal -> Quarter Wave Plate -> Vertical Analyzer')
plt.xlabel( "Quarter Waveplate angle (degrees)" )
plt.ylabel( "Transmission" )
plt.show()
```

## Crossed polarizers with half-wave plate between

Show that the total transmission of horizontally-polarized light that passes through a half-wave plate and then through a vertical linear analyzer is $4\sin^2\theta\cos^2\theta = \sin^2 2\theta$ and plot as the analyzer is rotated through 180 degrees.

```python
theta = sympy.Symbol('theta', real=True)

incident = sym_jones.field_horizontal()
plate = sym_jones.op_half_wave_plate(theta)
analyzer = sym_jones.op_linear_polarizer(sympy.pi/2)

result = analyzer * plate * incident

fraction = sym_jones.intensity(result)
fraction
```

$\displaystyle 4 \sin^{2}{\left(\theta \right)} \cos^{2}{\left(\theta \right)}$

```python
theta = np.linspace(0, 180, 500)

plt.plot(theta, np.sin(np.radians(2*theta))**2 )

plt.title('Horizontal -> Half Wave Plate -> Vertical Analyzer')
plt.xlabel( "Half waveplate Angle (degrees)" )
plt.ylabel( "Transmission" )
plt.show()
```

## Waveplates not at design wavelength

The phase shift in a halfwave plate depends on the wavelength. If the halfwave plates are designed for 532nm light, but we use them for 633nm light then how will the measurements be affected?  (Assume the speeds along the fast and slow axis are independent of wavelength).

Plot the transmitted light for both 633 and 532nm light.

```python
f = sympy.Symbol('f', real=True)
theta = sympy.Symbol('theta', real=True)
lambda_0 = sympy.Symbol('lambda_0', real=True)
lambda_1 = sympy.Symbol('lambda_1', real=True)
f = lambda_0/lambda_1

incident = sym_jones.field_horizontal()
plate_1 = sym_jones.op_retarder(theta,f*sympy.pi)
analyzer = sym_jones.op_linear_polarizer(sympy.pi/2)

result_1 = analyzer * plate_1 * incident

fraction_1 = sym_jones.intensity(result_1)

fraction_1
```

$\displaystyle 4 \sin^{2}{\left(\theta \right)} \sin^{2}{\left(\frac{\pi \lambda_{0}}{2 \lambda_{1}} \right)} \cos^{2}{\left(\theta \right)}$

```python
lambda_0 = 532e-9
lambda_1 = 633e-9
theta = np.linspace(0, 180, 500)
plt.plot(theta, np.sin(np.radians(2*theta))**2 ,label='532nm')
plt.plot(theta, np.sin(lambda_0*np.pi/2/lambda_1)**2*np.sin(np.radians(2*theta))**2, label='633nm' )

plt.title('Horizontal -> Half Wave Plate -> Vertical Analyzer')
plt.xlabel( "Half waveplate Angle (degrees)" )
plt.ylabel( "Transmission" )
plt.legend()
plt.show()
```


$\newcommand{\bkt}[1]{\left(#1\right)}$
$\newcommand{\dsum}[1]{\displaystyle\sum}$
$\newcommand{\spade}{\bkt{\spadesuit}}$
$\newcommand{\club}{\bkt{\clubsuit}}$

Polynomial Interpolation
==

1.1 **Introduction**

Given the values of a function $f(x)$ at $n+1$ distinct locations of $x$, say $\{x_i\}_{i=0}^n$, we could approximate $f$ by a polynomial function $p_n(x)$ of degree $n$ that satisfies

$$p_n\bkt{x_i} = f\bkt{x_i}$$

We can construct the polynomial $p_n(x)$ as $p_n(x) = a_0 + a_1 x + a_2 x^2 + \cdots + a_n x^n$. The $n+1$ coefficients are determined by forcing $p_n(x)$ to pass through the data points. This leads to $n+1$ equations in $n+1$ unknowns, $a_0,a_1,\ldots,a_n$, i.e.,
$$y_i = a_0 + a_1 x_i + a_2 x_i^2 + \cdots + a_n x_i^n$$
for $i \in \{0,1,2,\ldots,n\}$. This procedure for finding the coefficients of the polynomial is not very attractive. It involves solving a linear system, whose matrix is extremely ill-conditioned. See below.

```python
import scipy as sp;
import numpy as np;

from scipy.stats import binom;
import matplotlib.pylab as pl;
from scipy import linalg
from numpy import linalg
from ipywidgets import interact;
```

```python
Nmax = 41;
N = np.arange(2,Nmax);
c = np.zeros(Nmax-2);
for n in N:
    x = np.linspace(-1,1,n);
    V = np.vander(x,increasing="True");
    c[n-2] = np.linalg.cond(V);

pl.semilogy(N,c,'k-+');
```

A better way to go about interpolating with polynomials is via Lagrange interpolation. Define the Lagrange polynomial $L_i(x)$ to be $1$ when $x=x_i$ and is zero at all the other nodes, i.e.,
$$L_i\bkt{x_j} = \delta_{ij}$$
We then have
$$p_n(x) = \sum_{i=0}^n f_i L_i(x)$$
Since we want $L_i(x)$ to vanish at all $x_j$, where $j \neq i$, we have $L_i(x) = c_i \prod_{j \neq i}\bkt{x-x_j}$. Further, since $L_i\bkt{x_i} = 1$, we get that $c = \dfrac1{\prod_{j \neq i} \bkt{x_i-x_j}}$. Hence, we see that
$$L_i\bkt{x} = \prod_{j \neq i} \bkt{\dfrac{x-x_j}{x_i-x_j}}$$
If we call $l_i(x) = \prod_{j \neq i} \bkt{x-x_j}$ and $w_i = \prod_{j \neq i} \bkt{\dfrac1{x_i-x_j}}$, we see that $L_i(x) = w_i l_i(x)$.

Further, if we set $l(x) = \prod_{j=0}^n \bkt{x-x_j}$, we see that $l_i(x) = \dfrac{l(x)}{x-x_i}$, and hence $L_i(x) = \dfrac{w_il(x)}{x-x_i}$ and hence we see that
\begin{align}
p_n(x) = l(x) \bkt{\sum_{i=0}^n \dfrac{w_i f_i}{x-x_i}} \,\,\, \spade
\end{align}
Note that $\spade$ is an attractive way to compute the Lagrange interpolant. It requires $\mathcal{O}\bkt{n^2}$ work to calculate $w_i$'s, followed by $\mathcal{O}(n)$ work to compute the interpolant for each $x$. This is called as the ***first form of Barycentric interpolation***.

What about updating when a new interpolation node $x_{n+1}$ is added? There are only two steps involved.

- For $i \in\{0,1,\ldots,n\}$, divide each $w_i$ by $\bkt{x_i-x_{n+1}}$. Cost is $n+1$ flops.
- Compute $w_{n+1}$ for another $n+1$ flops.

Hence, we see that the Lagrange interpolant can also be updated at $\mathcal{O}(n)$ flops.

The above barycentric formula can be made even more elegant in practice. Note that the function $1$ gets interpolated by any polynomial exactly. Hence, we see that
$$1 = l(x) \bkt{\sum_{i=0}^n \dfrac{w_i}{x-x_i}}$$
which gives us that
$$l(x) = \dfrac1{\bkt{\displaystyle\sum_{i=0}^n \dfrac{w_i}{x-x_i}}}$$
Hence, we obtain the ***second form of Barycentric interpolation***
\begin{align}
p_n(x) = \dfrac{\bkt{\displaystyle\sum_{i=0}^n \dfrac{w_i f_i}{x-x_i}}}{\bkt{\displaystyle\sum_{i=0}^n \dfrac{w_i}{x-x_i}}} \,\,\, \club
\end{align}
Again note that it is fairly easy to incorporate a new intepolation node at a cost of $\mathcal{O}(n)$. Below is the Lagrange interpolation using the original form.

```python
def function(x):
#     f = np.abs(x)+x/2-x**2;
    f = 1.0/(1+25*x*x);
#     f = np.abs(x+0.3) + np.abs(x-0.2) +  + np.abs(x*x*x*x-0.8);
    return f;

def Lagrange(xnodes,x,i):
    f = 1;
    nnodes = np.size(xnodes);
    for j in range(0,i):
        f = f*(x-xnodes[j])/(xnodes[i]-xnodes[j]);
    for j in range(i+1,nnodes):
        f = f*(x-xnodes[j])/(xnodes[i]-xnodes[j]);
    return f;

def Chebyshev(nnodes,xplot):
    # Chebyshev node interpolation
    xnodes    = np.cos(np.arange(0,nnodes)*np.pi/(nnodes-1));
    fnodes     = function(xnodes);
    fplot        = 0;
    for i in range(0,nnodes):
        fplot = fplot + fnodes[i]*Lagrange(xnodes,xplot,i);
    return xnodes, fnodes, fplot;

def Uniform(nnodes,xplot):
    # Uniform node interpolation
    xnodes    = np.linspace(-1,1,nnodes);
    fnodes     = function(xnodes);
    fplot        = 0;
    for i in range(0,nnodes):
        fplot = fplot + fnodes[i]*Lagrange(xnodes,xplot,i);
    return xnodes, fnodes, fplot;

def Bernstein(n,xplot):
    xnodes = np.linspace(-1,1,nnodes);
    fnodes  = function(xnodes);
    fplot = 0;
    for i in range(0,nnodes):
        fplot = fplot + sp.stats.binom.pmf(i,nnodes-1,0.5+0.5*xplot)*function(xnodes[i]);
    return fplot;

nplot       = 1001;
xplot       = np.linspace(-1,1,nplot);
f_actual   = function(xplot);
@interact
def inter(nnodes=(5,45,2)):
    xnodes, fnodes, fplot = Chebyshev(nnodes,xplot);
#     xnodes, fnodes, fplot = Uniform(nnodes,xplot);
    # fplot        = Bernstein(nnodes,xplot);

    error       = f_actual-fplot;
    print(np.amax(np.abs(error)))
    pl.plot(xplot,f_actual,'-');
    pl.plot(xplot,fplot,'r');
    pl.rcParams["figure.figsize"] = [16,4];
```

    interactive(children=(IntSlider(value=25, description='nnodes', max=45, min=5, step=2), Output()), _dom_classe…

```python

```


# Singular value decomposition (SVD)

The singular value decompostion of a real-valued $m \times n$ matrix $\boldsymbol{A}$ is:

$$
\boldsymbol{A} = \boldsymbol{U} \boldsymbol{\Sigma} \boldsymbol{V}^{T}
$$

where

- $\boldsymbol{U}$ is an $m \times m$ orthogonal matrix;
- $\boldsymbol{\Sigma}$ is an $m \times n$ diagonal matrix with diagonal entries $\sigma_{1} \ge  \sigma_{2} \ge \ldots \ge \sigma_{p} \ge 0$, where $p = \min(m, n)$; and
- $\boldsymbol{U}$ is an $n \times n$ orthogonal matrix.

We will use NumPy to compute the SVD and Matplotlib to visualise results, so we first import some modules:

```python
%matplotlib inline
import matplotlib
import matplotlib.pyplot as plt
import numpy as np

from PIL import Image
```

**Note:** If you run this notebook yourself it can take sometime because it computes number of moderate size SVD problems.

## Low rank approximations

Recall that we can represent a matrix as a sum of rank-1 matrices:

$$
\boldsymbol{A} = \sum_{i} \sigma_{i} \boldsymbol{u}_{i} \boldsymbol{v}^{T}_{i}
$$

where $\sigma_{i}$ is the $i$th singular value and $\boldsymbol{u}_{i}$ and $\boldsymbol{v}_{i}$ are the $i$th columns vectors of $\boldsymbol{U}$ and $\boldsymbol{V}$, respectively from the SVD. Clearly, for any $\sigma_{i} = 0$ we can avoid storing the data that makes no contribution. If $\sigma_{i}$ is small, then the contribution of $\boldsymbol{u}_{i} \boldsymbol{v}^{T}_{i}$ is small and we discard it and introduce only a small 'error' to the matrix. We will use low rank approximations in a number of examples in this notebook.

## Data compression

We start with a $100 \times 200$ matrix that has entries equal to one or zero. We create a matrix with all entries set to zero, and we then set some entries equal to one in the pattern of rectangle.

```python
A = np.ones((100, 200))
A[33:33 + 4, 33:133] = 0.0
A[78:78 + 4, 33:133] = 0.0
A[33:78+4, 33:33+4] = 0.0
A[33:78+4, 129:129+4] = 0.0
plt.imshow(A, cmap='gray', interpolation='none')
plt.show()
```

Performing the SVD and counting the number of singular values that are greater than $10^{-9}$:

```python
U, s, V = np.linalg.svd(A, full_matrices=False)
print("Number of singular values greater than 1.0e-9: {}".format((s > 1.0e-9).sum()))
```

With only three nonzero singular values, we could reconstruct the matrix with very little data - just three singular values and six vectors.

### Removing noise

We consider the same matrix problem again, this time with some back ground noise in the white regions.

```python
A = np.ones((100, 200))
A = A - 1.0e-1*np.random.rand(100, 200)
A[33:33 + 4, 33:133] = 0.0
A[78:78 + 4, 33:133] = 0.0
A[33:78+4, 33:33+4] = 0.0
A[33:78+4, 129:129+4] = 0.0
plt.imshow(A, cmap='gray', interpolation='none');
```

The effect of the noise is clear in the image.

We can try to eliminate much of the background noise via a low-rank approximation of the noisy image that discards information associated with small singular values of the matrix.

```python
# Compute SVD of nois matrix
U, s, V = np.linalg.svd(A, full_matrices=False)

# Set any singular values less than 1.0 equation zero
s[s < 1.0] = 0.0

# Reconstruct low rank approximation and display
A_denoised = np.dot(U, np.dot(np.diag(s), V))
plt.imshow(A_denoised, cmap='gray', interpolation='none')
plt.show();
```

We can see that much of the noise in the image has been eliminated.

## Image compression

### Gray scale image

We load a colour PNG file. It uses three colour channels (red/green/blue), with at each pixel an 8-bit unsigned integer (in the range $[0, 255]$, but sometimes represented as a float) for each colour for the colour intensity. This is know as 24-bit colour - three channels times 8 bit.

We load the image as three matrices (red, green, blue), each with dimension equal to the number pixels in each direction:

```python
from urllib.request import urlopen
url = "https://github.com/garth-wells/notebooks-3M1/raw/master/photo/2020-1.png"
img_colour = Image.open(urlopen(url))
img_colour = img_colour.convert('RGB')

print("Image size (pixels):", img_colour.size)
print("Image array shape:  ", np.array(img_colour).shape)

plt.figure(figsize=(15, 15/1.77))
plt.imshow(img_colour);
```

We could work with the colour image, but it is simpler to work with a gray scale image because then we have only one value for the colour intensity at each pixel rather than three (red/green/blue).

```python
img_bw = img_colour.convert('L')

plt.figure(figsize=(15, 15/1.77))
plt.imshow(img_bw, cmap='gray');
print("Image array shape: {}".format(img_bw.size))

plt.savefig("bw.pdf")
```

We can convert the image to a regular matrix with values between 0 and 255, with each entry corresponding to a pixel in the image. Creating the matrix and inspecting first four rows and three columns (top left corner of the image):

```python
img_array = np.array(img_bw)
print("Image shape:", img_array.shape)
print(img_array[:4, :3])
```

Now, maybe we can discard information associated with small singular values without perceiving any visual change in the image. To explore this, we compute the SVD of the gray scale image:

```python
U, s, V = np.linalg.svd(img_array, full_matrices=False)
```

The argument `full_matrices=False` tells NumPy to not store all the redundant zero terms in the $\boldsymbol{\Sigma}$ array. This is the normal approach in practice, but not in most text books. Note that NumPy return the singular values as a one-dimendional array, not as a matrix.

We now print the largest and smallest singular values, and plot all the singular values $\sigma_{i}$ on a log-scale:

```python
print("Number of singular values: {}".format(len(s)))
print("Max, min singular values: {}, {}".format(s[0], s[-1]))

plt.xlabel('$i$')
plt.ylabel('$\sigma_i$')
plt.title('Singular values')
plt.yscale('log')
plt.plot(s, 'bo');

plt.savefig("bw-svd.pdf")
```

We can now try compressing the image. We first try retaining using only the largest 25% of values:

```python
# Compute num_sigma/4 (25%) and zero values
r = int(0.25*len(s))

# Re-construct low rank approximation (this may look a little cryptic, but we use the below
# expression to avoid unecessary computation)
compressed = U[:,:r].dot(s[:r, np.newaxis]*V[:r,:])
compressed = compressed.astype(int)

# Plot compressed and original image
fig, axes = plt.subplots(nrows=1, ncols=2, figsize=(18, 18/1.77));
axes[0].set_title('Compressed image with largest 25% of singular values retained')
axes[0].imshow(compressed, cmap='gray');
axes[1].set_title('Original image')
axes[1].imshow(img_array, cmap='gray');
```

We have discarded 3/4 of the singular values, but can  barely perceive a difference in the image.

To explore other levels of compression, we write a function that takes the fraction of singular values we wish to retain:

```python
def compress_image(U, s, V, f):
    "Compress image where 0 < f <= 1 is the fraction on singular values to retain"
    r = int(f*len(s))
    return (U[:,:r].dot(s[:r, np.newaxis]*V[:r,:])).astype(int)
```

Let's try retaining just 10% of the singular values:

```python
# Compress image/matrix
compressed = compress_image(U, s, V, 0.1)

# Plot compressed and original image
fig, axes = plt.subplots(nrows=1, ncols=2, figsize=(20, 20/1.77))
axes[0].set_title('Compressed image with largest 10% of singular values retained')
axes[0].imshow(compressed, cmap='gray');
axes[1].set_title('Original image')
axes[1].imshow(img_array, cmap='gray');

plt.savefig("bw-0-10.pdf")
```

Even with only 10% if the singular values retains, it is hard to perceive a difference between the images. Next we try keeping only 2%:

```python
# Compress image/matrix
compressed = compress_image(U, s, V, 0.02)

# Plot compressed and original image
fig, axes = plt.subplots(nrows=1, ncols=2, figsize=(20, 20/1.77))
axes[0].set_title('Compressed image with largest 2% of singular values retained')
axes[0].imshow(compressed, cmap='gray');
axes[1].set_title('Original image')
axes[1].imshow(img_array, cmap='gray');

plt.savefig("bw-0-02.pdf")
```

We now see some image clear degradation, but the image is sill recognisable. We'll try one more case where we retain only 0.5% of the singular values.

```python
# Compress image/matrix
compressed = compress_image(U, s, V, 0.005)

# Plot compressed and original image
fig, axes = plt.subplots(nrows=1, ncols=2, figsize=(20, 20/1.77))
axes[0].set_title('Compressed image with largest 0.5% of singular values retained')
axes[0].imshow(compressed, cmap='gray');
axes[1].set_title('Original image')
axes[1].imshow(img_array, cmap='gray');

plt.savefig("bw-0-005.pdf")
```

The image quality is now quite poor.

### Colour image: RGB

We'll now try compressing a colour image.

```python
print("Image array shape: {}".format(img_colour.size))

plt.figure(figsize=(20,20/1.77))
plt.title('This is a photo of 2020 3M1 class members')
plt.imshow(img_colour);
```

We can extract the red, green and blue components to have a look:

```python
# Display red, green and blue channels by zeroing other channels
fig, axes = plt.subplots(nrows=1, ncols=3, figsize=(20, 20/1.77))
img_array = np.array(img_colour)

# Zero the g/b channels
red = img_array.copy()
red[:,:,(1,2)] = 0.0
axes[0].imshow(red);

# Zero the r/b channels
green = img_array.copy()
green[:,:,(0,2)] = 0.0
axes[1].imshow(green);

# Zero the r/g channels
blue = img_array.copy()
blue[:,:,(0,1)] = 0.0
axes[2].imshow(blue);
```

We now compute an SVD for the matrix of each colour:

```python
# Compute SVD for each colour
U, s, V = [0]*3, [0]*3, [0]*3
for i in range(3):
    U[i], s[i], V[i] = np.linalg.svd(img_array[:, :, i], full_matrices=False)
```

Compressing the matrix for each colouring separately and then reconstructing the three-dimensional array:

```python
# Compress each colour separately
compressed = [compress_image(U[i], s[i], V[i], 0.1) for i in range(3)]

# Reconstruct 3D RGB array and filter any values outside of (0, 1)
compressed = np.dstack(compressed)
```

Comparing the compressed and original images side-by-side:

```python
fig, axes = plt.subplots(nrows=1, ncols=2, figsize=(20, 20/1.77))
axes[0].set_title('Image with largest 10% of singular values retained')
axes[0].imshow(compressed, interpolation="nearest");
axes[1].set_title('Original image')
axes[1].imshow(img_colour);
```

Retaining 10% of the singular values for each colour, we can see some artifacts in the compressed image, which indicates that using the SVD for each colour independently is probably not a good idea.

### Colour image: YCbCr

A better approach is to split the image into [YCbCr](https://en.wikipedia.org/wiki/YCbCr), rather than RGB.
YCbCr is splits the image into luminance (Y), and chrominance (Cb and Cr) colour values.

```python
img_colour_ycbcr = np.array(img_colour.convert("YCbCr"))
```

```python
# Display Luminance(Y), Blue Chroma(Cb) and Red Chroma(Cr) channels
fig, axes = plt.subplots(nrows=1, ncols=3, figsize=(20, 20/1.77))

Y = img_colour_ycbcr[:,:,0]
axes[0].imshow(Y, cmap='gray');

Cb = img_colour_ycbcr[:,:,1]
axes[1].imshow(Cb, cmap='gray');

Cr = img_colour_ycbcr[:,:,2]
axes[2].imshow(Cr, cmap='gray');
```

Compute the SVD of each channel:

```python
# Compute SVD for each channel
U, s, V = [0]*3, [0]*3, [0]*3
for i in range(3):
    U[i], s[i], V[i] = np.linalg.svd(img_colour_ycbcr[:, :, i], full_matrices=False)
```

Compress each channel, and display compressed channels in gray scale:

```python
# Compress each component separately
compressed = [compress_image(U[0], s[0], V[0], 0.05),
              compress_image(U[1], s[1], V[1], 0.005),
              compress_image(U[2], s[2], V[2], 0.005)]
# Reconstruct 3D YCbCr array
compressed = np.dstack(compressed)
fig, axes = plt.subplots(nrows=1, ncols=3, figsize=(20, 20/1.77))
Y = compressed[:,:,0]
axes[0].imshow(Y, cmap='gray');

Cb = compressed[:,:,1]
axes[1].imshow(Cb, cmap='gray');

Cr = compressed[:,:,2]
axes[2].imshow(Cr, cmap='gray');
```

Combine compressed channels:

```python
fig, axes = plt.subplots(nrows=1, ncols=2, figsize=(20, 20/1.77))
axes[0].set_title('Image with largest 20% of brightness singular values retained and 0.5% colours')
im = Image.fromarray(np.uint8(compressed), mode="YCbCr")
axes[0].imshow(im)
axes[1].set_title('Original image')
axes[1].imshow(img_colour);
```

### Interactive compression

We'll now create an interactive image with sliders to interactively control the compression level.

```python
from ipywidgets import widgets
from ipywidgets import interact

url = "https://github.com/garth-wells/notebooks-3M1/raw/master/photo/IMG_20190117_141222563.png"
img = Image.open(urlopen(url))
img_colour_ycbcr = np.array(img.convert("YCbCr"))

# Compute SVD for each channel
U0, s0, V0 = [0]*3, [0]*3, [0]*3
for i in range(3):
    U0[i], s0[i], V0[i] = np.linalg.svd(img_colour_ycbcr[:, :, i], full_matrices=False)

@interact(ratio_Y=(0.005, 0.4, 0.02),
          ratio_Cb=(0.001, 0.1, 0.01),
          ratio_Cr=(0.001, 0.1, 0.01))
def plot_image(ratio_Y=0.1, ratio_Cb=0.01, ratio_Cr=0.01):

    compressed = [compress_image(U0[0], s0[0], V0[0], ratio_Y),
                  compress_image(U0[1], s0[1], V0[1], ratio_Cb),
                  compress_image(U0[2], s0[2], V0[2], ratio_Cr)]

    # Reconstruct 3D YCbCr array
    compressed = np.dstack(compressed)
    img_compressed = Image.fromarray(np.uint8(compressed), mode="YCbCr")

    # Show
    fig, axes = plt.subplots(nrows=1, ncols=2, figsize=(20, 20/1.77))

    axes[0].set_title('Compressed image')
    axes[0].imshow(img_compressed)

    axes[1].set_title('Original image')
    axes[1].imshow(img)
```

## Effective rank

Determining the rank of a matrix is not a binary question in the context of floating point arithmetic or measurement errors. The SVD can be used to determine the 'effective rank' of a matrix.

Consider the matrix:

```python
A = np.array([[1, 1, 1], [2, 2, 2], [1, 0 ,1]])
print(A)
```

Clearly the first two rows are linearly dependent and the rank of this matrix is 2. We can verify this using NumPy:

```python
print("Rank of A is: {}".format(np.linalg.matrix_rank(A)))
```

We now add some noise in the range $(0, 10^{-6})$ to the matrix entries:

```python
np.random.seed(10)
A = A + 1.0e-6*np.random.rand(A.shape[0], A.shape[1])
```

We now test the rank:

```python
print("Rank of A (with noise) is: {}".format(np.linalg.matrix_rank(A)))
```

The problem is that we have a 'data set' that is linearly dependent, but this is being masked by very small measurement noise.

Computing the SVD of the matrix with noise and printing the singular values:

```python
U, s, V = np.linalg.svd(A)
print("The singular values of A (with noise) are: {}".format(s))
```

If we define the effective rank as the number of singular values that are greater than the noise level, the effective rank of $\boldsymbol{A}$ is 2.

## Rank deficient least-squares problems

For least squares problem, we have seen before that we solve

$$
\boldsymbol{A}^{T} \boldsymbol{A} \hat{\boldsymbol{x}} = \boldsymbol{A}^{T} \boldsymbol{b}
$$

or

$$
\begin{align}
\hat{\boldsymbol{x}} &= (\boldsymbol{A}^{T} \boldsymbol{A})^{-1} \boldsymbol{A}^{T} \boldsymbol{b}
\\
&= \boldsymbol{A}^{+}\boldsymbol{b}
\end{align}
$$

Everything is fine as long as $\boldsymbol{A}$ is full rank. The problem is that we might have data that leads to $\boldsymbol{A}$ not being full rank. For example, if we try to fit a polynomial in $x$ and $y$, but the data lies on a line.

We have covered in the lectures how to handle least-squares problems that are rank deficient. Here we present an example.

### Example: fitting points in a two-dimensional space

Say we are given four data points that depend on $x$ and $y$, and we are asked to fit a polynomial of the form

$$
f(x, y) = c_{00} + c_{10}x + c_{01}y + c_{11}xy
$$

to the data points. Normally, we would expect to be able to fit the above polynomial to four data points by interpolation, i.e. solving $\boldsymbol{A} \boldsymbol{c} = \boldsymbol{f}$ where
$\boldsymbol{A}$ a square Vandermonde matrix. However, if the points happened to lie on a line, then $\boldsymbol{A}$ will be singular. If the points happen to almost lie on a line, then $\boldsymbol{A}$ will be close to singular.

A possibility is to exclude zero or small singular values from the process, thereby finding a least-squares fit with minimal $\|\boldsymbol{c}\|_{2}$. We test this for the data set

\begin{equation}
f_{1}(1, 0) = 3, \\
f_{2}(2, 0) = 5, \\
f_{3}(3, 0) = 7, \\
f_{4}(4, 0) = 9.
\end{equation}

The data lies on the line $y = 0$, and is in fact is linear in $x$.

We create arrays to hold this data, and visualise the points:

```python
x, y, f = np.zeros(4), np.zeros(4), np.zeros(4)
x[0], y[0], f[0] = 1.0, 0.0, 3.0
x[1], y[1], f[1] = 2.0, 0.0, 5.0
x[2], y[2], f[2] = 3.0, 0.0, 7.0
x[3], y[3], f[3] = 4.0, 0.0, 9.0

from mpl_toolkits.mplot3d import Axes3D
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
ax.set_xlabel('x')
ax.set_ylabel('y')
ax.set_zlabel('f')
ax.scatter(x, y, f)
plt.show()
```

To find the polynomial coefficients we want to solve

\begin{equation}
\begin{bmatrix}
1 & x_{1} & y_{1} & x_{1}y_{1}  \\
1 & x_{2} & y_{2} & x_{2}y_{2}  \\
1 & x_{3} & y_{3} & x_{3}y_{3}  \\
1 & x_{4} & y_{4} & x_{4}y_{4}  \\
\end{bmatrix}
\begin{bmatrix}
c_{00} \\ c_{10} \\ c_{01} \\ c_{11}
\end{bmatrix}
=
\begin{bmatrix}
f_{1} \\ f_{2} \\ f_{3} \\ f_{4}
\end{bmatrix}
\end{equation}

where the matrix is the Vandermonde matrix. We can use a NumPy function to create the Vandermonde matrix:

```python
A = np.polynomial.polynomial.polyvander2d(y, x, [1, 1])
print(A)
```

It is clear by inspection that $\boldsymbol{A}$ is not full rank, and is rank 2.

Computing the SVD of $\boldsymbol{A}$ and printing the singular values:

```python
U, s, V = np.linalg.svd(A)
print(s)
```

We can see that two of the singular values are zero. To find a least-squares fit to the data with minimal $\| \boldsymbol{c}\|_{2}$ we compute

$$
\hat{\boldsymbol{c}} = \boldsymbol{V}_{1} \boldsymbol{\Sigma}^{+}
\boldsymbol{U}_{1}^{T}\boldsymbol{b}
$$

Creating $\boldsymbol{V}_{1}$,  $\boldsymbol{\Sigma}^{+}$ and $\boldsymbol{U}_{1}$ (recall that the NumPy SVD returns $\boldsymbol{V}^{T}$ rather than  $\boldsymbol{V}$):

```python
# Create view of U with last two columns removed
U1 = U[:, :2]

# Create view of V with last two columns removed
V1 = V[:2,:]

# Create Sigma^{+} by inverting the nonzero singular values and
# discarding the zero singular values
S1 = np.diag(1.0/s[:-2])
print(S1)
```

Computing the least-squares solution from $\hat{\boldsymbol{c}} = \boldsymbol{V}_{1} \boldsymbol{\Sigma}^{+} \boldsymbol{U}_{1}^{T}\boldsymbol{b}$:

```python
c = np.transpose(V1).dot(S1.dot(U1.T).dot(f))
print(c)
```

The solution is $f(x, y) = 1 + 2x$, which in this case in fact interpolates the data points. Plotting the function, we have a plane that passes through the points.

```python
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')

# Plot points
ax.set_xlabel('$x$')
ax.set_ylabel('$y$')
ax.set_zlabel('$f$')
ax.scatter(x, y, f)

# Plot surface
X = np.arange(0, 5, 0.2)
Y = np.arange(-5, 5, 0.2)
X, Y = np.meshgrid(X, Y)
Z = 1.0 + 2.0*X + 0.0*Y
surf = ax.plot_surface(X, Y, Z, rstride=5, cstride=5,  alpha=0.1)
ax.view_init(elev=30, azim=80)
plt.show()
```

We now try adding some noise to the sample positions and the measured values. The Vandermonde matrix is no longer singular so we can solve $\boldsymbol{A} \boldsymbol{c} = \boldsymbol{f}$ to get the polynomial coefficients:

```python
np.random.seed(20)
xn = x + 1.0e-3*(1.0 - np.random.rand(len(x)))
yn = y + 1.0e-3*(1.0 - np.random.rand(len(y)))
fn = f + 1.0e-3*(1.0 - np.random.rand(len(f)))

A = np.polynomial.polynomial.polyvander2d(yn, xn, [1, 1])
c = np.linalg.solve(A, fn)
print(c)
```

We now see significant coefficients for the $y$ and $xy$ terms in the interpolating polynomial just as a consequence of adding small amount of noise. Plotting the surface and the points, we see in dramatic impact of the noise.

```python
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')

# Plot points
ax.set_xlabel('$x$')
ax.set_ylabel('$y$')
ax.set_zlabel('$f$')
ax.scatter(xn, yn, fn)

# Plot surface
X = np.arange(0, 5, 0.2)
Y = np.arange(-5, 5, 0.2)
X, Y = np.meshgrid(X, Y)
Z = c[0] + c[1]*X + c[2]*Y + c[3]*X*Y
surf = ax.plot_surface(X, Y, Z, rstride=5, cstride=5,  alpha=0.1)
ax.view_init(elev=30, azim=80)
plt.show()
```

Performing an SVD on the matrix with noise and printing the singular values:

```python
U, s, V = np.linalg.svd(A)
print(s)
```

We see that two of the values are considerably small than the others. If we set these to zero and follow the least-squares procedure for rank-deficient problem:

```python
# Create view of U with last two columns removed
U1 = U[:, :2]

# Create view of V with last two columns removed
V1 = V[:2,:]

# Create \Sigma^{+}
S1 = np.diag(1.0/s[:-2])

c = np.transpose(V1).dot(S1.dot(U1.T).dot(f))
print(c)
```

We see that the fitting polynomial is very close to the noise-free case.

## Principal component analysis

Principal component analysis finds a transformation such that covariance of a data set is zero in the transformed directions, and the variance in these directions is greatest. From a dataset this tells us which are the 'important' parameters in a system.

Consider taking $N = 200$ measurements of two quantities $x_{1}$ and $x_{2}$. We model the system by:

```python
np.random.seed(1)
x0 = np.random.randn(200) + 5.0
x1 = 1.5*x0 + np.random.rand(len(x0))

ax = plt.axes()
ax.scatter(x0, x1, alpha=0.5);
ax.set_xlabel('$x_{1}$');
ax.set_ylabel('$x_{2}$');
```

We collect the data in a $200 \times 2$ matrix $\boldsymbol{X}$ (200 measurements, 2 variables):

```python
X = np.column_stack((x0, x1))
```

We can compute the covariance matrix $\boldsymbol{C}$ by making the columns of $\boldsymbol{X}$ zero mean and computing $\boldsymbol{X}^{T}\boldsymbol{X}^{T}/(N-1)$

```python
for c in range(X.shape[1]):
    X[:,c] = X[:,c] - np.mean(X[:,c])
C = (X.T).dot(X)/(len(x0)-1.0)
```

The covariance matrix is square and symmetric, so w can diagonalise it by computing the eigenvalues and eigenvectors.

We could also compute the SVD of $\boldsymbol{X}$ since the $\boldsymbol{V}$ is made of the eigenvectors of $\boldsymbol{X}^{T}\boldsymbol{X}^{T}$:

```python
U, s, V = np.linalg.svd(C)
print(s)
```

Plotting the data set and the principal directions:

```python
ax = plt.axes()
ax.set_aspect(1.0);
ax.set_ylim(-4.0, 4.0);
ax.set_xlabel('$x_{1}$')
ax.set_ylabel('$x_{2}$')
ax.quiver(V[0, 0], V[0, 1], angles='xy',scale_units='xy',scale=0.3);
ax.quiver(V[1, 0], V[1, 1], angles='xy',scale_units='xy',scale=1);
ax.scatter(X[:,0], X[:,1], alpha=0.2);
```

PCA effectively detects correlation in a data set. In the above example it suggest that the system could be modelled with one variable in the direction of the first column of $\boldsymbol{V}$.