[Sascha Spors](https://orcid.org/0000-0001-7225-9992),
Professorship Signal Theory and Digital Signal Processing,
[Institute of Communications Engineering (INT)](https://www.int.uni-rostock.de/),
Faculty of Computer Science and Electrical Engineering (IEF),
[University of Rostock, Germany](https://www.uni-rostock.de/en/)

# Tutorial Signals and Systems (Signal- und Systemtheorie)

Summer Semester 2021 (Bachelor Course #24015)

- lecture: https://github.com/spatialaudio/signals-and-systems-lecture
- tutorial: https://github.com/spatialaudio/signals-and-systems-exercises

WIP...
The project is currently under heavy development while adding new material for the summer semester 2021

Feel free to contact lecturer [frank.schultz@uni-rostock.de](https://orcid.org/0000-0002-3010-0294)

## Übung / Exercise 9 / 10

```python
import matplotlib.pyplot as plt
import numpy as np
import os
import sys
from scipy.signal import tf2zpk, bilinear
```

```python
cur_fol = os.getcwd()
print(cur_fol)
sys.path.append(cur_fol + '/../')
```

```python
from sig_sys_tools import plot_dtlti_analysis
```

# Analysis of Discrete-Time, Linear, Time-Invariant System (DTLTI)

Evaluate and plot the

* impulse response
* step response
* frequency response (level, phase, group delay)
* zero/pole/gain mapping, i.e. z-plane plot

of a DTLTI system given as z-transfer function

\begin{equation}
H(z) = \frac{\sum\limits_{n=0}^{N} b_n z^{-n}}{\sum\limits_{m=0}^{M} a_m z^{-m}}=\frac{b_0 + b_1 z^{-1} + b_2 z^{-2} + ...}{a_0 + a_1 z^{-1} + a_2 z^{-2} + ...}
\end{equation}

using `scipy.signal` routines for discrete-time domain signal processing. The coefficients $b_0, b_1,b_2,...,b_N$ are stored in vector/array b, the coefficients $a_0, a_1,a_2,...,a_M$ are stored in vector/array a.

Most often $H(z)$ is normalized such that $a_0=1$.

Note that `signal.dlti` handling has a known issue (https://dsp.stackexchange.com/questions/40312/why-does-pythons-scipy-signal-dimpulse-introduce-delay-in-impulse-response).
For workaround, **we must ensure** that **a and b** have the **same length** by suitable zero padding.
This is tedious for a long FIR.
In the context of this notebook for didactical purpose, this handling might be acceptable.
If analysis of long FIRs is needed, we might want to implement a optimized DFT-based handling on our own, until this issue might be fixed in `scipy.signal`.

Used abbreviations:

- DFT...discrete Fourier transform

- DTFT...discrete-time Fourier transform

- FIR...finite impulse response, i.e. we actually refer to a non-recursive system

- IIR...infinite impulse response, i.e. we actually refer to a recursive system

- DTLTI...discrete-time, linear, time-invariant

- ROC...region of convergence for z-transform.

Since we aim at causal impulse responses, ROC must be $|z|>\mathrm{max}(|z_{\infty}|)$.
Thus, if ROC (white) includes the unit circle (along which DTFT is defined), this implies that all poles are within the unit circle and by that that the causal system response is stable.

## Evaluating and Plotting Routine

The plotting routine `plot_dtlti_analysis(z, p, k, fs=1, Nf=2**10, Nt=2**5)` is found in the `sig_sys_tools.py`.

We demonstrate frequency axis handling with

* logarithmic x-axis along f / Hz for level, top, red
* linear x-axis along $\frac{\Omega}{2\pi} = \frac{f}{f_s}$ for level, bottom, blue
* linear x-axis along $\Omega$ for phase
* linear x-axis along $\frac{\Omega}{\pi}$ for group delay

in the subplots below.
We should get familiar with these different styles, since they often occur and have pros / cons in different applications / visualization strategies.
The choice of which response maps to which handling is rather arbitrary and we can freely switch to whatever style is best suitable.

We discuss systems that were analytically evaluated in the tutorials. Feel free to play around with other systems that you find in textbooks, online material or that come up by your own.

## Pass Thru = Digital Wire

\begin{equation}
H(z) = 1
\end{equation}

```python
# pass through
b = [+1]
a = [+1]
z, p, k = tf2zpk(b, a)
plot_dtlti_analysis(z, p, k)
plt.savefig('PassThru.pdf')
```

## Exercise 7.3: System H1

\begin{align}
H(z) = \frac{z^3 +z^2 -z + \frac{1}{2}}{z^3} = 1 + z^{-1} - z^{-2} + \frac{1}{2} z^{-3}
\end{align}

```python
# FIR filter, finite impulse response vector h would be handled as
b = [1, 1, -1, 1/2]  # = h
a = [1, 0, 0, 0]  # len(a)==len(b) handling :-(
z, p, k = tf2zpk(b, a)
plot_dtlti_analysis(z, p, k)
plt.savefig('System_UE7_3_H1.pdf')
```

## Exercise 7.3: System H3

\begin{align}
H(z) = \frac{2 z^2 + 1}{z^2-\frac{1}{2} z} = \frac{2 + z^{-2}}{1-\frac{1}{2} z^{-1}}
\end{align}

```python
# IIR filter
b = [2, 0, 1]
a = [1, -1/2, 0]  # note the sign reversal of coef a1 compared to block diagram
z, p, k = tf2zpk(b, a)
plot_dtlti_analysis(z, p, k)
plt.savefig('System_UE7_3_H3.pdf')
```

## Exercise 94A7A6D9E9

\begin{equation}
H(z) = \frac{z^2-z+2}{z^2-\frac{1}{2} z + \frac{1}{4}}=\frac{1-z^{-1}+2 z^{-2}}{1-\frac{1}{2} z^{-1} + \frac{1}{4} z^{-2}}
\end{equation}

```python
# IIR filter
b = [+1, -1, +2]
# note the sign reversal of coefs a1, a2 compared to block diagram
a = [+1, -1/2, +1/4]
z, p, k = tf2zpk(b, a)
plot_dtlti_analysis(z, p, k)
plt.savefig('system_94A7A6D9E9.pdf')
```

## Exercise 64BE42BAEF

\begin{equation}
H(z) =1 + z^{-2} + \frac{17}{4} z^{-4} + \frac{17}{4} z^{-6} + z^{-8} + z^{-10}
\end{equation}

```python
# FIR filter
b = np.array([1, 0, 1, 0, 17/4, 0, 17/4, 0, 1, 0, 1])
a = np.zeros_like(b)
a[0] = 1
z, p, k = tf2zpk(b, a)
plot_dtlti_analysis(z, p, k)
```

## Discrete-Time Version of a Laplace-Domain Transfer Function

The analog lowpass filter of 2nd order from

https://github.com/spatialaudio/signals-and-systems-exercises/blob/master/laplace_system_analysis/lowpass2nd_44EB4169E9.ipynb

\begin{equation}
H(s) = \frac{1}{\frac{16}{25} s^2 + \frac{24}{25} s + 1}
\end{equation}

is transfered to discrete-time domain by setting the sampling frequency 100 times above the cutoff frequency and using the so called bilinear transform. This design exhibits a zero on the unit circle at $z=-1$ for lowpass characteristics.

The 2nd order differential equation leads to a 2nd order difference equation, which can be interpreted as a 2nd order recursive filter, usually referred to as infinite impulse response (IIR) filter.

```python
# digital filter design with so called bilinear transform of a
# continuous-time ODE (we will learn this in detail in the DSP course):
# ODE RLC-example from 'solving_2nd_order_ode.pdf' /
# 'frequency_response_2nd_order_ode.pdf' is
# 16/25 y''(t) + 24/25 y'(t) + y(t) = DiracDelta(t), y'(t-=0)=0, y(t-=0)=0
# for example sampled with
# sampling frequency in Hz, note: omega0 = 5/4
fs = np.ceil((5/4)/(2*np.pi)*100)
# note that we just round up to integer for nicer plotting
print('fs = ', fs, 'Hz')
[b, a] = bilinear([25/16], [1, 24/16, 25/16], fs=fs)
print('b = ', b)
print('a = ', a)

z, p, k = tf2zpk(b, a)
plot_dtlti_analysis(z, p, k, fs=fs, Nt=np.int32(np.ceil(fs*10)))

# we obtain a discrete-time 2nd order IIR-filter with lowpass characteristics
plt.savefig('AnalogODE_Bilinear.pdf')
```

## Copyright

This tutorial is provided as Open Educational Resource (OER), to be found at
https://github.com/spatialaudio/signals-and-systems-exercises
accompanying the OER lecture
https://github.com/spatialaudio/signals-and-systems-lecture.
Both are licensed under a) the Creative Commons Attribution 4.0 International
License for text and graphics and b) the MIT License for source code.
Please attribute material from the tutorial as *Frank Schultz,
Continuous- and Discrete-Time Signals and Systems - A Tutorial Featuring
Computational Examples, University of Rostock* with
``main file, github URL, commit number and/or version tag, year``.


# CF Part 4 - SVD method
> Collaborative Filtering on MovieLens Latest-small Part 4 - Finding recommendations using model based singular value decomposition (SVD) method

- toc: true
- badges: true
- comments: true
- categories: [movie, collaborative]
- image:

Due to the high level sparsity of the rating matrix $R$, **user-based** and **item-based** collaborative filtering suffer from **data sparsity** and **scalability**. These cause user and item-based collaborative filtering to be less effective and highly affect their performences.

To address the high level sparsity problem, [Sarwar et al. (2000)](http://files.grouplens.org/papers/webKDD00.pdf) proposed to reduce the dimensionality of the rating $R$ using the *Singular Value Decomposition (SVD)* algorithm.

---

### How do SVD works ?

As described is Figure 1, SVD factors the rating matrix $R$ of size $m\times n$ into three matrices $P$, $\Sigma$ and $Q$ as follows :

\begin{equation}
R = P\Sigma Q^{\top}.
\end{equation}

Here, $P$ and $Q$ are two orthogonal matrices of size $m\times \hat{k}$ and $n\times \hat{k}$ respectively and $\Sigma$ is a diagonal matrix of size $ \hat{k}\times \hat{k}$ (with $ \hat{k}$ the rank of matrix $R$) having all singular values of the rating matrix R as its diagonal entries ([Billsus and Pazzani, 1998](https://www.ics.uci.edu/~pazzani/Publications/MLC98.pdf), [Sarwar et al. (2000)](http://files.grouplens.org/papers/webKDD00.pdf)).

 Figure 1 : Singular value decomposition of rating matrix $R$

After having choosen $k$, the dimension of factors that will represent users and items, we can truncate matrix $\Sigma$ by only retaining its $k$ largest singular values to yield $\Sigma_k$ and reduce matrices $P$ and $Q$ accordingly to obtain $P_k$ and $Q_k$. The rating matrix will then be estimated as

\begin{equation}
R_k = P_k\Sigma_k Q_k^{\top}.
\end{equation}

Once these matrices are known, they can be used for rating predictions ant top-N recommendations. $P_k\Sigma_k^{\frac{1}{2}}$ represents the latent space of users and $\Sigma_k^{\frac{1}{2}}Q_k^{\top}$ the latent space of items. Rating prediction for user $u$ on $i$ is done by the following formular

\begin{equation}
\hat{R}_{u,i} = \begin{bmatrix}P_k\Sigma_k^{\frac{1}{2}}\end{bmatrix}_u \begin{bmatrix}\Sigma_k^{\frac{1}{2}}Q_k^{\top}\end{bmatrix}_i.
\end{equation}

Before applying SVD, its important to fill in missing values of the rating matrix $R$. [Sarwar et al. (2000)](http://files.grouplens.org/papers/webKDD00.pdf) found the item’s mean rating to be useful default values. The user's average rating can also be used but the former shown better performances. Adding ratings normalization by subtracting the user mean rating or other baseline predictor can improve accuracy.

---

### SVD algorithm

> 1. Factor the normalize rating matrix $R_{norm}$ to obtain matrices $P$, $\Sigma$ and $Q$
> 2. Reduce $\Sigma$ to dimension $k$ to obtain $\Sigma_k$
> 3. Compute the square-root of $\Sigma_k$ to obtain $\Sigma_k^{\frac{1}{2}}$
> 4. Compute the resultant matrices $P_k\Sigma_k^{\frac{1}{2}}$ and $\Sigma_k^{\frac{1}{2}}Q_k^{\top}$ that will be used to compute recommendation scores for any user and items.

---

### Implementation details

SVD can easily be implemented using python library such as ```numpy```, ```scipy``` or ```sklearn```. As described by Andrew Ng in his [Machine Learning course](https://www.coursera.org/learn/machine-learning/lecture/CEXN0/vectorization-low-rank-matrix-factorization), it's not recommended to implement the standard SVD by ourselves. Instead, we can take advantage of matrix libraries (such as those listed before) that are optimized for matrix computations and vectorization.

Now let's implement the SVD collaborative filtering

### Download useful tools

```python
import os

if not (os.path.exists("recsys.zip") or os.path.exists("recsys")):
    !wget https://github.com/nzhinusoftcm/review-on-collaborative-filtering/raw/master/recsys.zip
    !unzip recsys.zip
```

### Import requirements

```
matplotlib==3.2.2
numpy==1.19.2
pandas==1.0.5
python==3.7
scikit-learn==0.24.1
scikit-surprise==1.1.1
scipy==1.6.2
```

```python
from recsys.datasets import mlLastedSmall, ml100k, ml1m
from sklearn.preprocessing import LabelEncoder
from scipy.sparse import csr_matrix

import pandas as pd
import numpy as np
import os
```

### Loading movielen ratings

```python
ratings, movies = mlLastedSmall.load()
```

Let's see how our rating matrix looks like

```python
pd.crosstab(ratings.userid, ratings.itemid, ratings.rating, aggfunc=sum)
```

      itemid
      1
      2
      3
      4
      5
      6
      7
      8
      9
      10
      ...
      193565
      193567
      193571
      193573
      193579
      193581
      193583
      193585
      193587
      193609

      userid

      1
      4.0
      NaN
      4.0
      NaN
      NaN
      4.0
      NaN
      NaN
      NaN
      NaN
      ...
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN

      2
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      ...
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN

      3
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      ...
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN

      4
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      ...
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN

      5
      4.0
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      ...
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN

      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...

      606
      2.5
      NaN
      NaN
      NaN
      NaN
      NaN
      2.5
      NaN
      NaN
      NaN
      ...
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN

      607
      4.0
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      ...
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN

      608
      2.5
      2.0
      2.0
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      4.0
      ...
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN

      609
      3.0
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      4.0
      ...
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN

      610
      5.0
      NaN
      NaN
      NaN
      NaN
      5.0
      NaN
      NaN
      NaN
      NaN
      ...
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN

610 rows × 9724 columns

We can observe that our rating matrix has many of unobserved value. However, as we described earlier, the SVD algorithm requires that all inputs in the matrix must be defined. Let's initialize the unobserved ratings with item's average that led to better performances compared to the user's average or even a null initialization ([Sarwar et al. (2000)](http://files.grouplens.org/papers/webKDD00.pdf)).

We can go further and subtrat from each rating the corresponding user mean to normalize the data. This helps to improve the accuracy of the model.

```python
# get user's mean rating
umean = ratings.groupby(by='userid')['rating'].mean()
```

```python
def rating_matrix(ratings):
    """
    1. Fill NaN values with item's average ratings
    2. Normalize ratings by subtracting user's mean ratings

    :param ratings : DataFrame of ratings data
    :return
        - R : Numpy array of normalized ratings
        - df : DataFrame of normalized ratings
    """

    # fill missing values with item's average ratings
    df = pd.crosstab(ratings.userid, ratings.itemid, ratings.rating, aggfunc=sum)
    df = df.fillna(df.mean(axis=0))

    # subtract user's mean ratings to normalize data
    df = df.subtract(umean, axis=0)

    # convert our dataframe to numpy array
    R = df.to_numpy()

    return R, df

# generate rating matrix by calling function rating_matrix
R, df = rating_matrix(ratings)
```

$R$ is our final rating matrix. This is how the final rating matrix looks like

```python
df
```

      itemid
      1
      2
      3
      4
      5
      6
      7
      8
      9
      10
      ...
      193565
      193567
      193571
      193573
      193579
      193581
      193583
      193585
      193587
      193609

      userid

      1
      -0.366379
      -0.934561
      -0.366379
      -2.009236
      -1.294951
      -0.366379
      -1.181194
      -1.491379
      -1.241379
      -0.870167
      ...
      -0.866379
      -1.366379
      -0.366379
      -0.366379
      -0.866379
      -0.366379
      -0.866379
      -0.866379
      -0.866379
      -0.366379

      2
      -0.027346
      -0.516458
      -0.688660
      -1.591133
      -0.876847
      -0.002197
      -0.763091
      -1.073276
      -0.823276
      -0.452064
      ...
      -0.448276
      -0.948276
      0.051724
      0.051724
      -0.448276
      0.051724
      -0.448276
      -0.448276
      -0.448276
      0.051724

      3
      1.485033
      0.995921
      0.823718
      -0.078755
      0.635531
      1.510181
      0.749288
      0.439103
      0.689103
      1.060315
      ...
      1.064103
      0.564103
      1.564103
      1.564103
      1.064103
      1.564103
      1.064103
      1.064103
      1.064103
      1.564103

      4
      0.365375
      -0.123737
      -0.295940
      -1.198413
      -0.484127
      0.390523
      -0.370370
      -0.680556
      -0.430556
      -0.059343
      ...
      -0.055556
      -0.555556
      0.444444
      0.444444
      -0.055556
      0.444444
      -0.055556
      -0.055556
      -0.055556
      0.444444

      5
      0.363636
      -0.204545
      -0.376748
      -1.279221
      -0.564935
      0.309715
      -0.451178
      -0.761364
      -0.511364
      -0.140152
      ...
      -0.136364
      -0.636364
      0.363636
      0.363636
      -0.136364
      0.363636
      -0.136364
      -0.136364
      -0.136364
      0.363636

      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...

      606
      -1.157399
      -0.225581
      -0.397784
      -1.300256
      -0.585971
      0.288679
      -1.157399
      -0.782399
      -0.532399
      -0.161187
      ...
      -0.157399
      -0.657399
      0.342601
      0.342601
      -0.157399
      0.342601
      -0.157399
      -0.157399
      -0.157399
      0.342601

      607
      0.213904
      -0.354278
      -0.526481
      -1.428953
      -0.714668
      0.159982
      -0.600911
      -0.911096
      -0.661096
      -0.289884
      ...
      -0.286096
      -0.786096
      0.213904
      0.213904
      -0.286096
      0.213904
      -0.286096
      -0.286096
      -0.286096
      0.213904

      608
      -0.634176
      -1.134176
      -1.134176
      -0.777033
      -0.062747
      0.811903
      0.051009
      -0.259176
      -0.009176
      0.865824
      ...
      0.365824
      -0.134176
      0.865824
      0.865824
      0.365824
      0.865824
      0.365824
      0.365824
      0.365824
      0.865824

      609
      -0.270270
      0.161548
      -0.010655
      -0.913127
      -0.198842
      0.675808
      -0.085085
      -0.395270
      -0.145270
      0.729730
      ...
      0.229730
      -0.270270
      0.729730
      0.729730
      0.229730
      0.729730
      0.229730
      0.229730
      0.229730
      0.729730

      610
      1.311444
      -0.256738
      -0.428941
      -1.331413
      -0.617127
      1.311444
      -0.503371
      -0.813556
      -0.563556
      -0.192344
      ...
      -0.188556
      -0.688556
      0.311444
      0.311444
      -0.188556
      0.311444
      -0.188556
      -0.188556
      -0.188556
      0.311444

610 rows × 9724 columns

### Ids encoding

Let's encode users and items ids such that their values range from 0 to 909 (for users) and from 0 to 9723 (for items)

```python
users = sorted(ratings['userid'].unique())
items = sorted(ratings['itemid'].unique())

# create our id encoders
uencoder = LabelEncoder()
iencoder = LabelEncoder()

# fit our label encoder
uencoder.fit(users)
iencoder.fit(items)
```

    LabelEncoder()

### SVD Algorithm

Now that our rating data has been normalize and that missing values has been filled, we can apply the SVD algorithm. Several libraries may be useful such as ```numpy```, ```scipy```, ```sklearn```, ... Let's try it with ```numpy```.

In our SVD class we provide the following function :

1. ```fit()``` : compute the svd of the rating matrix and save the resultant matrices P, S and Qh (Q transpose) as attributs of the SVD class.
2. ```predict()```: use matrices P, S and Qh to make ratin prediction for a given $u$ user on an item $i$. Computations are made over encoded values of userid and itemid. The predicted value is the dot product between $u^{th}$ row of $P.\sqrt{S}$ and the $i^{th}$ column of $\sqrt{S}.Qh$. **Note** that since we normalized rating before applying SVD, the predicted value will also be normalize. So, to get the final predicted rating, we have to add to the predicted value the mean rating of user $u$.
3. ```recommend()```: use matrices P, S and Qh to make recommendations to a given user. The recommended items are those that where not rated by the user and received a high score according to the svd model.

```python
class SVD:

    def __init__(self, umeam):
        """
        :param
            - umean : mean ratings of users
        """
        self.umean = umean.to_numpy()

        # init svd resultant matrices
        self.P = np.array([])
        self.S = np.array([])
        self.Qh = np.array([])

        # init users and items latent factors
        self.u_factors = np.array([])
        self.i_factors = np.array([])

    def fit(self, R):
        """
        Fit the SVD model with rating matrix R
        """
        P, s, Qh = np.linalg.svd(R, full_matrices=False)

        self.P = P
        self.S = np.diag(s)
        self.Qh = Qh

        # latent factors of users (u_factors) and items (i_factors)
        self.u_factors = np.dot(self.P, np.sqrt(self.S))
        self.i_factors = np.dot(np.sqrt(self.S), self.Qh)

    def predict(self, userid, itemid):
        """
        Make rating prediction for a given user on an item

        :param
            - userid : user's id
            - itemid : item's id

        :return
            - r_hat : predicted rating
        """
        # encode user and item ids
        u = uencoder.transform([userid])[0]
        i = iencoder.transform([itemid])[0]

        # the predicted rating is the dot product between the uth row
        # of u_factors and the ith column of i_factors
        r_hat = np.dot(self.u_factors[u,:], self.i_factors[:,i])

        # add the mean rating of user u to the predicted value
        r_hat += self.umean[u]

        return r_hat

    def recommend(self, userid):
        """
        :param
            - userid : user's id
        """
        # encode user
        u = uencoder.transform([userid])[0]

        # the dot product between the uth row of u_factors and i_factors returns
        # the predicted value for user u on all items
        predictions = np.dot(self.u_factors[u,:], self.i_factors) + self.umean[u]

        # sort item ids in decreasing order of predictions
        top_idx = np.flip(np.argsort(predictions))

        # decode indices to get their corresponding itemids
        top_items = iencoder.inverse_transform(top_idx)

        # sorted predictions
        preds = predictions[top_idx]

        return top_items, preds

```

Now let's create our SVD model and provide to it user's mean rating; Fit the model with the normalized rating matrix $R$.

```python
# create our svd model
svd = SVD(umean)

# fit our model with normalized ratings
svd.fit(R)
```

### Rating prediction

Our model has been fitted.
OK ...

Let's make some predictions for users using function ```predict``` of our SVD class. Here are some truth ratings

```python
ratings.head(10)
```

      userid
      itemid
      rating
      timestamp

      0
      1
      1
      4.0
      964982703

      1
      1
      3
      4.0
      964981247

      2
      1
      6
      4.0
      964982224

      3
      1
      47
      5.0
      964983815

      4
      1
      50
      5.0
      964982931

      5
      1
      70
      3.0
      964982400

      6
      1
      101
      5.0
      964980868

      7
      1
      110
      4.0
      964982176

      8
      1
      151
      5.0
      964984041

      9
      1
      157
      5.0
      964984100

Let's apply our model to make see if our predictions make sens. We will make predictions for user 1 on the 10 items listed above.

```python
# user for which we make predictions
userid = 1

# list of items for which we are making predictions for user 1
items = [1,3,6,47,50,70,101,110,151,157]

# predictions
for itemid in items:
    r = svd.predict(userid=userid, itemid=itemid)
    print('prediction for userid={} and itemid={} : {}'.format(userid, itemid, r))
```

    prediction for userid=1 and itemid=1 : 4.000000000000007
    prediction for userid=1 and itemid=3 : 4.0000000000000036
    prediction for userid=1 and itemid=6 : 3.9999999999999885
    prediction for userid=1 and itemid=47 : 5.000000000000003
    prediction for userid=1 and itemid=50 : 4.9999999999999964
    prediction for userid=1 and itemid=70 : 2.9999999999999893
    prediction for userid=1 and itemid=101 : 5.000000000000002
    prediction for userid=1 and itemid=110 : 3.999999999999993
    prediction for userid=1 and itemid=151 : 5.000000000000007
    prediction for userid=1 and itemid=157 : 5.000000000000019

Our prediction error is less than 0.00001

### Make recommendations

The ```recommend``` function makes recommendations for a given user.

```python
userid = 1

# items sorted in decreasing order of predictions for user 1
sorted_items, preds = svd.recommend(userid=userid)

##
# Now let's exclud from that sorted list items already purchased by the user
##

# list of items rated by the user
uitems = ratings.loc[ratings.userid == userid].itemid.to_list()

# remove from sorted_items items already in uitems and pick the top 30 ones
# as recommendation list
top30 = np.setdiff1d(sorted_items, uitems, assume_unique=True)[:30]

# get corresponding predictions from the top30 items
top30_idx = list(np.where(sorted_items == idx)[0][0] for idx in top30)
top30_predictions = preds[top30_idx]

# find corresponding movie titles
zipped_top30 = list(zip(top30,top30_predictions))
top30 = pd.DataFrame(zipped_top30, columns=['itemid','predictions'])
List = pd.merge(top30, movies, on='itemid', how='inner')

# show the list
List
```

      itemid
      predictions
      title
      genres

      0
      140627
      5.0
      Battle For Sevastopol (2015)
      Drama|Romance|War

      1
      27373
      5.0
      61* (2001)
      Drama

      2
      115727
      5.0
      Crippled Avengers (Can que) (Return of the 5 D...
      Action|Adventure

      3
      151769
      5.0
      Three from Prostokvashino (1978)
      Animation

      4
      4402
      5.0
      Dr. Goldfoot and the Bikini Machine (1965)
      Comedy

      5
      166183
      5.0
      Junior and Karlson (1968)
      Adventure|Animation|Children

      6
      165959
      5.0
      Alesha Popovich and Tugarin the Dragon (2004)
      Animation|Comedy|Drama

      7
      147328
      5.0
      The Adventures of Sherlock Holmes and Dr. Wats...
      Crime

      8
      128087
      5.0
      Trinity and Sartana Are Coming (1972)
      Comedy|Western

      9
      69860
      5.0
      Eichmann (2007)
      Drama|War

      10
      131610
      5.0
      Willy/Milly (1986)
      Comedy|Fantasy

      11
      69211
      5.0
      Boy Eats Girl (2005)
      Comedy|Horror

      12
      3531
      5.0
      All the Vermeers in New York (1990)
      Comedy|Drama|Romance

      13
      113829
      5.0
      One I Love, The (2014)
      Comedy|Drama|Romance

      14
      69469
      5.0
      Garfield's Pet Force (2009)
      Animation

      15
      26928
      5.0
      Summer's Tale, A (Conte d'Ã©tÃ©) (1996)
      Comedy|Drama|Romance

      16
      131724
      5.0
      The Jinx: The Life and Deaths of Robert Durst ...
      Documentary

      17
      26849
      5.0
      Stand, The (1994)
      Adventure|Drama|Fantasy|Horror|Sci-Fi

      18
      114265
      5.0
      Laggies (2014)
      Comedy|Romance

      19
      3096
      5.0
      My Man Godfrey (1957)
      Comedy

      20
      70451
      5.0
      Max Manus (2008)
      Action|Drama|War

      21
      26840
      5.0
      Sonatine (Sonachine) (1993)
      Action|Comedy|Crime|Drama

      22
      4788
      5.0
      Moscow Does Not Believe in Tears (Moskva sleza...
      Drama|Romance

      23
      71268
      5.0
      Tyler Perry's I Can Do Bad All by Myself (2009)
      Comedy|Drama

      24
      7122
      5.0
      King of Hearts (1966)
      Comedy|Drama|War

      25
      4813
      5.0
      When Worlds Collide (1951)
      Sci-Fi

      26
      67618
      5.0
      Strictly Sexual (2008)
      Comedy|Drama|Romance

      27
      3567
      5.0
      Bossa Nova (2000)
      Comedy|Drama|Romance

      28
      131098
      5.0
      Saving Santa (2013)
      Animation|Children|Comedy

      29
      131130
      5.0
      Tom and Jerry: A Nutcracker Tale (2007)
      Animation|Comedy

The first 30 items have an equivalent rating prediction for the user 1

### Improving memory based collaborative filtering

SVD can be applied to improve user and item-based collaborative filtering. Instead of computing similarities between user's or item's ratings, we can represent users and items by their corresponding latent factors extracted from the SVD algorithm.

### Matrix Factorization

The **Matrix Factorization** algorithm is a variant of SVD. Also known as Regularized SVD, it uses the *Gradient Descent* optimizer to optimize the cost function while training the model.

[Go to](https://github.com/nzhinusoftcm/review-on-collaborative-filtering/blob/master/5.Matrix_Factorization.ipynb) the Matrix Factorization variant of SVD.

## Reference

1. Daniel Billsus  and  Michael J. Pazzani (1998). [Learning Collaborative Information Filters](https://www.ics.uci.edu/~pazzani/Publications/MLC98.pdf)
2. Sarwar et al. (2000). [Application of Dimensionality Reduction in Recommender System -- A Case Study](http://files.grouplens.org/papers/webKDD00.pdf)

## Author

[Carmel WENGA](https://www.linkedin.com/in/carmel-wenga-871876178/),

PhD student at Université de la Polynésie Française,

Applied Machine Learning Research Engineer,

[ShoppingList](https://shoppinglist.cm), NzhinuSoft.


```python
import numpy as np
import sympy as sy
import control.matlab as cm
sy.init_printing()
```

```python
h = 0.14 # Sampling period
z = sy.symbols('z', real=False)
r1,r2, s0,s1,s2 = sy.symbols('r1,r2,s0,s1,s2', real=True) # Second order controller
pc1 = -1-1j
pc2 = np.conjugate(pc1)
pc3 = -1
pd1 = np.exp(h*pc1)
pd2 = np.exp(h*pc2)
pd3 = np.exp(h*pc3)

# The desired closed-loop poles
Ac = sy.poly(sy.simplify((z-pd1)*(z-pd2))*(z-pd3), z)
Ao = sy.poly(z**2, z)

# The plant
Bp = sy.poly(h**3/6.0 * (z**2 + 4*z + 1), z)
Ap = sy.poly((z-1)**3, z)

# The controller
Rp = sy.poly(z**2 + r1*z + r2, z)
Sp = sy.poly(s0*z**2 + s1*z + s2, z)
LHS = Ap*Rp + Bp*Sp
RHS = Ac*Ao
dioph=(LHS-RHS).all_coeffs()
```

```python
sy.simplify((z-pd1)*(z-pd2))
```

```python
RHS.all_coeffs()
```

```python

```

```python
sol=sy.solve(dioph, (r1,r2,s0,s1, s2))
print sol[r1]
print sol[r2]
print sol[s0]
print sol[s1]
print sol[s2]

t0 = Ac.evalf(subs={z:1})/Bp.evalf(subs={z:1,})
print t0
```

    0.329165114241075
    0.0680324802533507
    174.428108085880
    -321.564681193252
    148.759067609367
    1.62249450199559

```python
type(t0)
```

    sympy.core.numbers.Float

```python
G = cm.tf([1], [1, 0, 0,0])
Gd = cm.c2d(G, h)
Fb = cm.tf([float(sol[s0]), float(sol[s1]), float(sol[s2])],
           [1., float(sol[r1]), float(sol[r2])], h)
Ff = cm.tf([float(t0), 0, 0], [1., float(sol[r1]), float(sol[r2])], h)

Gc = Ff * cm.feedback(Gd, Fb)
cm.pole(Gc)
```

    array([  8.69358235e-01 +0.00000000e+00j,
             8.60852431e-01 +1.21312956e-01j,
             8.60852431e-01 -1.21312956e-01j,
            -1.64582557e-01 +2.02348862e-01j,
            -1.64582557e-01 -2.02348862e-01j,
             5.95862269e-14 +4.54381191e-07j,   5.95862269e-14 -4.54381191e-07j])

```python
print A2p
print A2p.evalf(subs={z:1})
print Bp
print Bp.evalf(subs={z:1})

```

    Poly(z**2 - 1.58555290309441*z - (-0.792776451547206 + 0.168974310731771*I)*(0.792776451547206 + 0.168974310731771*I), z, domain='EX')
    0.0714939167206446
    Poly(0.00133942860759726*z + 0.00124712240506047, z, domain='RR')
    0.00258655101265772

```python
0.3/(5*np.sqrt(2))
```

    0.042426406871192847

```python
np.exp(-0.21)*np.sin(0.21)
```

    0.16897431073177133

```python
np.exp(0.03*(-14))
```

    0.65704681981505675

```python

```


```python
import numpy as np
import pandas as pd
import linearsolve as ls
import matplotlib.pyplot as plt
plt.style.use('classic')
%matplotlib inline
```

# Homework 8

**Instructions:** Complete the notebook below. Download the completed notebook in HTML format. Upload assignment using Canvas.

**Due:** Mar. 12 at **12:30pm.**

## Exercise: New-Keynesian Model Stochastic Simulation

### Equilibrium Conditions and Paramter Values

The most basic version of the New-Keynesian Model can be expressed as:

\begin{align}
y_t & = E_t y_{t+1}  - \left( r_{t} - \bar{r}\right) + g_t\\
i_{t} & = r_{t} + E_t \pi_{t+1}\\
i_{t} & = \bar{r} + \pi^T + \phi_{\pi}\big(\pi_t - \pi^T\big) + \phi_{y}\big(y_t - \bar{y}\big)\\
\pi_t -\pi^T & = \beta \left( E_t\pi_{t+1} - \pi^T\right)  + \kappa (y_t -\bar{y})+ u_t,
\end{align}

where: $y_t$ is (log) output, $r_t$ is the real interest rate, $i_t$ is the nominal interest rate, $\pi_t$ is the rate of inflation between periods $t-1$ and $t$, $\bar{r}$ is the long-run average real interest rate or the *natural rate of interest*, $\beta$ is the household's subjective discount factor, and $\pi^T$ is the central bank's inflation target. The coeffieints $\phi_{\pi}$ and $\phi_{y}$ reflect the degree of intensity to which the central bank *endogenously* adjusts the nominal interest rate in response to movements in inflation and output.

The variables $g_t$ and $u_t$ represent exogenous shocks to aggregate demand and inflation. They follow AR(1) processes:

\begin{align}
g_{t+1} & = \rho_g g_{t} + \epsilon^g_{t+1}\\
u_{t+1} & = \rho_u u_{t} + \epsilon^u_{t+1}
\end{align}

### Parameter Values:

You will use the following parameter values:

| $\bar{y}$ | $\beta$ | $\bar{r}$    | $\kappa$ | $\pi^T$ | $\phi_{\pi}$ | $\phi_y$ | $\rho_g$ | $\sigma_g^2$ | $\rho_u$ | $\sigma_u^2$ |
|-----------|---------|--------------|----------|---------|--------------|----------|----------|--------------|----------|--------------|
| 0         | 0.995   | $-\log\beta$ | 0.1      | 0.02/4  | 1.5          | 0.5/4    | 0.5      | 0.0022         | 0.5      | 0.0012        |

### Input Model and Solve

Refer to the Notebook for the week 9 discussion section for a complete example of how to input the model and solve.

```python
# Create a variable called 'parameters' that stores the model parameter values in a Pandas Series.
parameters = pd.Series()
parameters['y_bar'] = 0
parameters['beta'] = 0.995
parameters['r_bar'] = -np.log(parameters.beta)
parameters['kappa'] = 0.1
parameters['pi_T'] = 0.02/4
parameters['phi_pi'] = 1.5
parameters['phi_y'] = 0.5/4
parameters['rho_g'] = 0.5
parameters['rho_u'] = 0.5
parameters['sigma_g_2'] = 0.002**2
parameters['sigma_u_2'] = 0.001**2

# Create variable called 'var_names' that stores the variable names in a list with state variables ordered first
var_names = ['g','u','y','pi','i','r']

# Create variable called 'shock_names' that stores an exogenous shock name for each state variable.
shock_names = ['e_g','e_u']

# Define a function that evaluates the equilibrium conditions of the model solved for zero.
def equilibrium_equations(variables_forward,variables_current,parameters):

    # Parameters
    p = parameters

    # Current variables
    cur = variables_current

    # Forward variables
    fwd = variables_forward

    # IS equation
    is_equation = fwd.y - (cur.r -p.r_bar) + cur.g - cur.y

    # Fisher_equation
    fisher_equation = cur.r + fwd.pi - cur.i

    # Monetary policy
    monetary_policy = p.r_bar + p.pi_T + p.phi_pi*(cur.pi - p.pi_T) + p.phi_y*cur.y - cur.i

    # Phillips curve
    phillips_curve = p.beta*(fwd.pi- p.pi_T) + p.kappa*cur.y + cur.u - (cur.pi-p.pi_T)

    # Demand process
    demand_process = p.rho_g*cur.g - fwd.g

    # Inflation process
    inflation_process = p.rho_u*cur.u - fwd.u

    # Stack equilibrium conditions into a numpy array
    return np.array([
            is_equation,
            fisher_equation,
            monetary_policy,
            phillips_curve,
            demand_process,
            inflation_process
        ])

# Initialize the model into a variable named 'nk_model'
nk_model = ls.model(equations = equilibrium_equations,
                 n_states=2,
                 var_names=var_names,
                 shock_names=shock_names,
                 parameters = parameters)

# Compute the steady state numerically using .compute_ss() method of nk_model
guess = [0,0,0,0.01,0.01,0.01]
nk_model.compute_ss(guess)

# Approximate and solve the model. Set log_linear argument to False since the model is already linear
nk_model.approximate_and_solve(log_linear=False)
```

### Stochastic Simulation

Construct a stochastic simulation of the New-Keynesian model with the follwoing properties:

1. 201 periods.
2. Seed for random number generator: 126.

Note that the shock covariance matrix is:

\begin{align}
\text{Covariance matrix} & = \left[\begin{array}{cc}\sigma_g^2 & 0\\ 0 & \sigma_u^2\end{array}\right]
\end{align}

Use the following values for $\sigma_g^2$ and $\sigma_u^2$ in the simulation:

| $\sigma_g^2$      | $\sigma_u^2$      |
|-------------------|-------------------|
| 0.0022 | 0.0012 |

Refer to the Notebook for the week 7 discussion section for an example of how to use `linearsolve` to construct a stochastic simulation.

```python
# Compute the simulation
nk_model.stoch_sim(T=201,seed=126,cov_mat=[[parameters['sigma_g_2'],0],[0,parameters['sigma_u_2']]])
```

### Analyze Simulation Results

Construct a plot with simulated output and inflation plotted together. Multiply simulated output by 100 and simulated inflation by 400 since, by convention, we always annualize inflation and interest rates.

```python
# Create a figure with dimensions 12x4. PROVIDED
fig = plt.figure(figsize=(12,4))

# Create the left axis. PROVIDED
ax1 = fig.add_subplot(1,1,1)

# Plot the simulated series for output (times 100) and inflation (times 400)
(nk_model.simulated['y']*100).plot(ax=ax1,lw=3,alpha=0.75,grid=True)
(nk_model.simulated['pi']*400).plot(ax=ax1,lw=3,alpha=0.75,grid=True)

# Construct legend. PROVIDED
ax1.legend(loc='center left', bbox_to_anchor=(1, 0.5))
```

Compute the standard deviations of inflation $\pi_t$ (times 400), output $y_t$ (times 100), and the nominal interest rate $i_t$ (times 400) for the simulated series.

```python
print('pi:',round(nk_model.simulated[['pi']].std()[0]*400,5))
print('y: ',round(nk_model.simulated[['y']].std()[0]*100,5))
print('i: ',round(nk_model.simulated[['i']].std()[0]*400,5))
```

    pi: 0.74496
    y:  0.41778
    i:  1.03754

Compute the coefficients of correlation of inflation $\pi_t$, output $y_t$, and the nominal interest rate $i_t$ for the simulated series.

```python
nk_model.simulated[['pi','y','i']].corr()
```

      pi
      y
      i

      pi
      1.000000
      -0.462278
      0.983935

      y
      -0.462278
      1.000000
      -0.296543

      i
      0.983935
      -0.296543
      1.000000

**Questions**

1. In the smulation data, which variable fluctuates more, inflation (times 400) or output (times 100)?
2. Do the simulations suggest a positive or negative correlation between output and inflation?
3. Note that for the simulations, it was assumed that the variances on the inflation and demand shocks were about the same. Based on the answer to your previous question, which shocks appear to have a dominant effect on the dynamics of the model?

**Answers**

1. Inflation.
2. A negative correlation between output and inflation.
3. It appears that the inflation shocks dominate because the model implies that output and inflation are negatively correlated following inflation shocks but positively cerrelated following demand shocks. Since both shocks are occuring simultaneously, it's apparent that the inflation shocks dominate the demand shocks.


```python
def pow(x, n, I, mult):
    # https://sahandsaba.com/five-ways-to-calculate-fibonacci-numbers-with-python-code.html
    """
    Returns x to the power of n. Assumes I to be identity relative to the
    multiplication given by mult, and n to be a positive integer.
    """
    if n == 0:
        return I
    elif n == 1:
        return x
    else:
        y = pow(x, n // 2, I, mult)
        y = mult(y, y)
        if n % 2:
            y = mult(x, y)
        return y

```

```python
def identity_matrix(n):
    """Returns the n by n identity matrix."""
    r = list(range(n))
    return [[1 if i == j else 0 for i in r] for j in r]

```

```python
def matrix_multiply(A, B):
    BT = list(zip(*B))
    return [[sum(a * b
                 for a, b in zip(row_a, col_b))
            for col_b in BT]
            for row_a in A]
```

```python
def fib(n):
    F = pow([[1, 1], [1, 0]], n, identity_matrix(2), matrix_multiply)
    return F[0][1]

```

```python
# github.com/nsc9's code begins here
from sympy import *
from sympy.abc import x,y,z
fib2 = []
poly1 = []
poly2 = []
poly3 = []
poly4 = []
poly5 = []
poly6 = []
poly7 = []
for i in range(0,2000):
    #print(-fib(i),"---------------------------------------------------------------------------------------------------------",fib(i))
    fib2.append(fib(i))
    poly1.append(x+fib2[i])
    poly2.append(fib2[i]*x+fib2[i])
    poly3.append(fib2[i]*x**2+fib2[i]*x+fib2[i])
    poly4.append(fib2[i]*x**3+fib2[i]*x**2+fib2[i]*x+fib2[i])
    poly5.append(fib2[i]*x**4+fib2[i]*x**3+fib2[i]*x**2+fib2[i]*x+fib2[i])
    poly6.append(fib2[i]*x**5+fib2[i]*x**4+fib2[i]*x**3+fib2[i]*x**2+fib2[i]*x+fib2[i])
    poly7.append(fib2[i]*x**8+fib2[i]*x**7+fib2[i]*x**6+fib2[i]*x**5+fib2[i]*x**4+fib2[i]*x**3+fib2[i]*x**2+fib2[i]*x**1+fib2[i])
print(poly7[0:10])
eqs = []
for i in range(1,100,2):
    #print(i)
    #plot(poly3[i]/10)
    #print(poly3[i])
    eqs.append(poly7[i])
    #print("   ")
    #'--Fibonacci numbers------------------------------------------------------------',fib2,'--x + Fibnumbr------------------------------------------------------------ ',poly1,'--Fibnumr*x+Fibnumbr-----------------------------------------------------------------  ',poly2,'--Fibnumbr*x**2+Fibnumbrx+Fibnumbr-----------------------------------------------------------------  ',poly3
```

    [0, x**8 + x**7 + x**6 + x**5 + x**4 + x**3 + x**2 + x + 1, x**8 + x**7 + x**6 + x**5 + x**4 + x**3 + x**2 + x + 1, 2*x**8 + 2*x**7 + 2*x**6 + 2*x**5 + 2*x**4 + 2*x**3 + 2*x**2 + 2*x + 2, 3*x**8 + 3*x**7 + 3*x**6 + 3*x**5 + 3*x**4 + 3*x**3 + 3*x**2 + 3*x + 3, 5*x**8 + 5*x**7 + 5*x**6 + 5*x**5 + 5*x**4 + 5*x**3 + 5*x**2 + 5*x + 5, 8*x**8 + 8*x**7 + 8*x**6 + 8*x**5 + 8*x**4 + 8*x**3 + 8*x**2 + 8*x + 8, 13*x**8 + 13*x**7 + 13*x**6 + 13*x**5 + 13*x**4 + 13*x**3 + 13*x**2 + 13*x + 13, 21*x**8 + 21*x**7 + 21*x**6 + 21*x**5 + 21*x**4 + 21*x**3 + 21*x**2 + 21*x + 21, 34*x**8 + 34*x**7 + 34*x**6 + 34*x**5 + 34*x**4 + 34*x**3 + 34*x**2 + 34*x + 34]

```python
diff(x**x)
```

$\displaystyle x^{x} \left(\log{\left(x \right)} + 1\right)$

```python
poly7[11]
```

$\displaystyle 89 x^{8} + 89 x^{7} + 89 x^{6} + 89 x^{5} + 89 x^{4} + 89 x^{3} + 89 x^{2} + 89 x + 89$

```python
plot(eqs[0],eqs[2],eqs[3],eqs[4],eqs[5],eqs[6],eqs[7],eqs[8],eqs[9],E**x)
```

```python
plot(eqs[6]*0.85,E**x,legend = True)
```

```python
plot(eqs[6]*0.851,E**x,legend = True, xlim = (0,11),size = (13,4))
```

```python
plot(eqs[0],eqs[1],eqs[2],eqs[3],eqs[4],eqs[5],E**x,x**x,legend = True,size = (23,4),yscale='log',ylim=(0.1,10**11),xlim = (-10,10))
```

```python
from sympy import Sum
from sympy.abc import k,m
print(Sum(eqs[0], (x, 1, 10)).doit(), "=")
Sum(eqs[0], (x, 1, 10))

```

    188039796 =

$\displaystyle \sum_{x=1}^{10} \left(x^{8} + x^{7} + x^{6} + x^{5} + x^{4} + x^{3} + x^{2} + x + 1\right)$

```python
print(Sum(eqs[1], (x, 1, 10)).doit(), "=")
Sum(eqs[1], (x, 1, 10))
```

    376079592 =

$\displaystyle \sum_{x=1}^{10} \left(2 x^{8} + 2 x^{7} + 2 x^{6} + 2 x^{5} + 2 x^{4} + 2 x^{3} + 2 x^{2} + 2 x + 2\right)$

```python
print(Sum(eqs[2], (x, 1, 10)).doit(), "=")
Sum(eqs[2], (x, 1, 10))
```

    940198980 =

$\displaystyle \sum_{x=1}^{10} \left(5 x^{8} + 5 x^{7} + 5 x^{6} + 5 x^{5} + 5 x^{4} + 5 x^{3} + 5 x^{2} + 5 x + 5\right)$

```python
print(Sum(eqs[3], (x, 1, 10)).doit(), "=")
Sum(eqs[3], (x, 1, 10))
```

    2444517348 =

$\displaystyle \sum_{x=1}^{10} \left(13 x^{8} + 13 x^{7} + 13 x^{6} + 13 x^{5} + 13 x^{4} + 13 x^{3} + 13 x^{2} + 13 x + 13\right)$

```python
print(Sum(eqs[4], (x, 1, 10)).doit(), "=")
Sum(eqs[4], (x, 1, 10))
```

    6393353064 =

$\displaystyle \sum_{x=1}^{10} \left(34 x^{8} + 34 x^{7} + 34 x^{6} + 34 x^{5} + 34 x^{4} + 34 x^{3} + 34 x^{2} + 34 x + 34\right)$

```python
6393353064/2444517348
```

    2.6153846153846154

```python
2444517348/940198980
```

    2.6

```python
940198980/376079592
```

    2.5

```python
print(Sum(eqs[5], (x, 1, 10)).doit(), "=")
Sum(eqs[5], (x, 1, 10))
```

    16735541844 =

$\displaystyle \sum_{x=1}^{10} \left(89 x^{8} + 89 x^{7} + 89 x^{6} + 89 x^{5} + 89 x^{4} + 89 x^{3} + 89 x^{2} + 89 x + 89\right)$

```python
16735541844/6393353064
```

    2.6176470588235294

```python
Sum(eqs[5], (x, 1, 10)).doit()/Sum(eqs[5-1], (x, 1, 10)).doit()
```

$\displaystyle \frac{89}{34}$

```python
for i in range(1,25):
    print((Sum(eqs[i], (x, 1, i)).doit()/Sum(eqs[i-1], (x, 1, i)).doit()).evalf())
```

    2.00000000000000
    2.50000000000000
    2.60000000000000
    2.61538461538462
    2.61764705882353
    2.61797752808989
    2.61802575107296
    2.61803278688525
    2.61803381340013
    2.61803396316671
    2.61803398501736
    2.61803398820533
    2.61803398867044
    2.61803398873830
    2.61803398874820
    2.61803398874965
    2.61803398874986
    2.61803398874989
    2.61803398874989
    2.61803398874989
    2.61803398874989
    2.61803398874989
    2.61803398874989
    2.61803398874989

```python
for i in range(1,50):
    print(i,"   ",fib2[i+1]/fib2[i])
```

    1     1.0
    2     2.0
    3     1.5
    4     1.6666666666666667
    5     1.6
    6     1.625
    7     1.6153846153846154
    8     1.619047619047619
    9     1.6176470588235294
    10     1.6181818181818182
    11     1.6179775280898876
    12     1.6180555555555556
    13     1.6180257510729614
    14     1.6180371352785146
    15     1.618032786885246
    16     1.618034447821682
    17     1.6180338134001253
    18     1.618034055727554
    19     1.6180339631667064
    20     1.6180339985218033
    21     1.618033985017358
    22     1.6180339901755971
    23     1.618033988205325
    24     1.618033988957902
    25     1.6180339886704431
    26     1.6180339887802426
    27     1.618033988738303
    28     1.6180339887543225
    29     1.6180339887482036
    30     1.6180339887505408
    31     1.6180339887496482
    32     1.618033988749989
    33     1.618033988749859
    34     1.6180339887499087
    35     1.6180339887498896
    36     1.618033988749897
    37     1.618033988749894
    38     1.6180339887498951
    39     1.6180339887498947
    40     1.618033988749895
    41     1.618033988749895
    42     1.618033988749895
    43     1.618033988749895
    44     1.618033988749895
    45     1.618033988749895
    46     1.618033988749895
    47     1.618033988749895
    48     1.618033988749895
    49     1.618033988749895

```python
import matplotlib as mpl
import matplotlib.pyplot as plt
import numpy as np
fig, ax = plt.subplots()  # Create a figure containing a single axes.

digsumfibs = []
def sum_digits(n):
    #https://stackoverflow.com/questions/14939953/sum-the-digits-of-a-number
    s = 0
    while n:
        s += n % 10
        n //= 10
    return s

for i in range(0,98):
    print("The digit sum of fib number",fib2[i], "=",sum_digits(fib2[i]))
    digsumfibs.append(sum_digits(fib2[i]))
ax.plot([digsumfibs[0],digsumfibs[1],digsumfibs[2],digsumfibs[3],digsumfibs[4],digsumfibs[5],digsumfibs[6],digsumfibs[7],
         digsumfibs[8],digsumfibs[9]])
```

```python
fig, ax = plt.subplots()  # Create a figure containing a single axes.

digsumfibs = []
def sum_digits(n):
    #https://stackoverflow.com/questions/14939953/sum-the-digits-of-a-number
    s = 0
    while n:
        s += n % 10
        n //= 10
    return s

for i in range(0,98,1):
    #print("The digit sum of fib number",fib2[i], "=",sum_digits(fib2[i]))
    digsumfibs.append(sum_digits(fib2[i]))
ax.plot([digsumfibs[0],digsumfibs[1],digsumfibs[2],digsumfibs[3],digsumfibs[4],digsumfibs[5],digsumfibs[6],digsumfibs[7],
         digsumfibs[8],digsumfibs[9],digsumfibs[10],digsumfibs[11],digsumfibs[12],digsumfibs[13],digsumfibs[14],digsumfibs[15],
         digsumfibs[16],digsumfibs[17],digsumfibs[18],digsumfibs[19],digsumfibs[20],digsumfibs[21],digsumfibs[22],digsumfibs[23],digsumfibs[24],digsumfibs[25],digsumfibs[26],digsumfibs[27],
        digsumfibs[28],digsumfibs[29],digsumfibs[30],digsumfibs[31],digsumfibs[32],digsumfibs[33],digsumfibs[34],digsumfibs[35],digsumfibs[36],digsumfibs[37],digsumfibs[38],digsumfibs[39]
        ,digsumfibs[40],digsumfibs[41],digsumfibs[42],digsumfibs[43],digsumfibs[44],digsumfibs[45],digsumfibs[46],digsumfibs[47],digsumfibs[48],digsumfibs[49],digsumfibs[50],digsumfibs[51],digsumfibs[52]
        ,digsumfibs[53],digsumfibs[54],digsumfibs[55],digsumfibs[56],digsumfibs[57],digsumfibs[58],digsumfibs[59],digsumfibs[60],digsumfibs[61],digsumfibs[62],digsumfibs[63],digsumfibs[64],
        digsumfibs[65],digsumfibs[66],digsumfibs[67],digsumfibs[68],digsumfibs[69],digsumfibs[70],digsumfibs[71],digsumfibs[72],digsumfibs[73],digsumfibs[74],digsumfibs[75],digsumfibs[76]
        ,digsumfibs[77],digsumfibs[78],digsumfibs[79],digsumfibs[80],digsumfibs[81],digsumfibs[82],digsumfibs[83],digsumfibs[84],digsumfibs[85],digsumfibs[86],digsumfibs[87],digsumfibs[88],
        digsumfibs[89],digsumfibs[90],digsumfibs[91],digsumfibs[92],digsumfibs[93],digsumfibs[94],digsumfibs[95],digsumfibs[96],digsumfibs[97]])
```

```python

```

```python

```


# Müller Brown I

Throughout the tutorials, we are going to demonstrate how to use totally accurate pathway simulator (TAPS). The tutorials aims for the user who wants to understands basic codes of the TAPS, presumably familiar with scientific Python programing like `numpy` and basic knowledge of chemical reaction pathway. Structre of TAPS begins with `Paths` class. `Paths` object contains other classes such as `Cartesian`, coordinate representation with kinetic energy calculatior, `Model`, potential and static energy calculator, `PathFinder`, pathway optimization algorithm and `Database` where calculated image data are stored. The reason for having this hierarchy is to efficietly link each class when they need each other. For example, static calculation such as gradient, potential or hessian requires the coordinate information and sometimes the data of previous calculation. When the calculation is conducted, `paths` object are handover to the `Model` object so that `Model` object can use the `Cartesian` or `Database` when they needed. Calculating kinetic energy works similarly. One might need only coordinate information but sometimes previous data can be necessary, for example one might wants to estimate the inertia of the atomic system with given only two angles. In that case, calcualtion of inertia is conducted using the data in the `Database` model.

Müller Brown (MB) potential is great example for the start. MB is 2D model potential involves a few parameters used for test bench. This tutorial will construct a pathway on the MB potential and calculate the potential and forces at each image. Our mission here is to understand basic understanding of TAPS throught doing

1. Construction of a pathway
2. Potential calculation
3. Direct action optimization (DAO)

examples.

Initial and the last point of the Muller Brown potential are (-0.55822365, 1.44172582) and (0.6234994, 0.02803776) respectively. We are going to make simple linear pathway connecting to ends using numpy.

```python
import numpy as np
N = 300
x = np.linspace(-0.55822365, 0.6234994, N)
y = np.linspace(1.44172582, 0.02803776, N)
coords = np.array([x, y])
coords.shape
```

    (2, 300)

## Construction of a pathway

`Cartesian` contains `coords`, consecutive coordinate representation of images, `epoch`, time spend for transition between two states, and `unit`. `coords` should be $(D\times N)$ size array or $(3\times A \times N)$ array where $D$ is dimension, $N$ is the number of consecutive images (including both ends) and $A$ the number of atoms. Reason for $N$ is last index is because of the row-major ordring of `python`.

Creating `Cartesian` object is simply put the `numpy` array into `coords` instance.

```python
from taps.coords import Cartesian

coords = Cartesian(coords=coords)
print(coords.shape, coords.D, coords.N, coords.epoch)
```

    (2, 300) 2 300 3

<!---`Cartesian` is an array like object that can be treat as an `numpy` array.

 We are considering descrete example of a pathway, where atomic path are considered as a consecutive images. `coords` is coordinate having a shape $(D\times N)$ matrix where $D$ stands for dimension and $N$ stands for the number of consecutive images. Beginning....
-->

## Potential calculation
MB potential is given by

 $$ V\left(x,y\right) = \sum_{\mu=1}^{4}{A_\mu e^{a_\mu \left(x-x_\mu^0\right)^2 + b_\mu \left(x-x_\mu^0\right) \left(y-y_\mu^0\right) + c_\mu\left(y-y_\mu^0\right)^2}}$$

`Model` object in TAPS has a few pre-defined toy model you can test your own algorithm. If you wants to know the parameters or info about that specific model, type "?" such as

```python
from taps.models import MullerBrown
?MullerBrown
model = MullerBrown()
print(model.A)
```

    [-2.   -1.   -1.7   0.15]

    Init signature: MullerBrown(results=None, label=None, prj=None, _cache=None, unit='eV')
    Docstring:
    Muller Brown Potential

    .. math::

       \begin{equation}
       V\left(x,y\right) =
       \sum_{\mu=1}^{4}{A_\mu e^{a_\mu \left(x-x_\mu^0\right)^2
       + b_\mu \left(x-x_\mu^0\right) \left(y-y_\mu^0\right)
       + c_\mu\left(y-y_\mu^0\right)^2}}
       \end{equation}

    * Initial position = (-0.55822365, 1.44172582)

    * Final position = (0.6234994, 0.02803776)

    Parameters
    ----------
    A = np.array([-200, -100, -170, 15])
    a = np.array([-1, -1, -6.5, 0.7])
    b = np.array([0, 0, 11, 0.6])
    c = np.array([-10, -10, -6.5, 0.7])
    x0 = np.array([1, 0, -0.5, -1])
    y0 = np.array([0, 0.5, 1.5, 1])
    potential_unit = 'unitless'

    Example
    -------

    >>> import numpy as np
    >>> N = 300
    >>> x = np.linspace(-0.55822365, 0.6234994, N)
    >>> y = np.linspace(1.44172582, 0.02803776, N)
    >>> paths.coords = np.array([x, y])
    File:           ~/anaconda3/envs/py37/lib/python3.7/site-packages/taps/models/mullerbrown.py
    Type:           type
    Subclasses:

`Paths` class contains `cooords`, which for the class `Cartesian` and `model` where `Model` class are stored. To calculate the properties along the pathway, we need a wrapper that connects both `Model` and `Cartesian`. `Paths` is classs for that conviently move around each objects. Calculating potential, gradients and Hessian can be conducted by scripting

```python
paths.get_potential()
paths.get_gradients()
paths.get_hessian()
```

as a default, it calculates properties throughout whole consecutive images except both end . If one wants to calculate including both end, one can use the keyword `index`. Index takes the list of step numbers and calculates only on that step.

```python
from taps.paths import Paths

paths = Paths(coords=coords, model=model)
print(paths.get_potential(index=np.s_[5:10]))
print(paths.get_gradients(index=[1, 2, 3]).shape)
```

    [-1.44794167 -1.43962909 -1.42985987 -1.41866061 -1.40606162]
    (2, 3)

## Visualization

In a 2D model calcualtion case, calculation are assumed to be very light. Thus, visualization of the package try to show the properties not only along the pathway but also the potential energy surface around it.
`Plotter` object visualize coordinate automatically with PES around it. It is not critical for the reaction calculation but it gives you insight around it. By default, 3D pathway like atomic system doesn't do PES map calcualtion. It only gives you the potential, kinetic and total energy along the pathway. Viewing the `paths` is simply,

```python
from taps.visualize import view
view(paths)
```

It showed something, but Since MB potential is exponentially increase outside its boundary, automatic resizeing or map leveling doesn't help to understand its view. In order to visuallize correctly you need to manipulate all the parameters involing `plotter`. Fortunately, in this example, we will use pre-defined parameter set that set to focus on important properties. By just typing keyword `viewer`.

```python
view(paths, viewer='MullerBrown')
```

Manipulating the `Cartesian` can be done by manually change the number of the `coords` array. However, manipulating the pathway manually is very diffcult process than changing just a coordinate of some atomic system. If one wants to test a random pathway rather than a liner pathway, one can fluctuate the pathway by adjusting sine components of the `coords`. TAPS has built in fluctuation function that do just that. You can type
```python
paths.fluctuate()
```
and it will randomize its pathway. Keyword `fluctuation` adjust the amplitutde of its fluctuation.

```python
paths.fluctuate(fluctuation=1)
```

```python
view(paths, viewer='MullerBrown')
```

Randomizing a pathway not only gives you more pleasing look but also make a pathway more unbiased. It can test pathway optimization algorithm working in any pathway in a simpler manner.

## Direct Action Optimization

`PathFinder` classs perfoms pathway optimization. Here we use direct action optimization method (DAO) to minimize the action of the pathway. `DAO` optimizes the pathway by lowering the value of its action using `scipy`'s `minimize` package. All together, this requires parameters needed for the `DAO` and parameters for `scipy`.

Parameters required for `DAO`, you can type "?"
```python
from taps.pathfinder import DAO
?DAO
```

```python
from taps.pathfinder import DAO
?DAO
```

    Init signature:
    DAO(
        action_kwargs=None,
        search_kwargs=None,
        use_grad=True,
        logfile=None,
        **kwargs,
    )
    Docstring:
    Direct action optimizer (DAO)

    Parameters
    ----------

    use_grad: bool , default True
        Whether to use graidnet form of the action.

    search_kwargs: Dict,
        Optimization keyword for scipy.minimize

    action_kwargs: Dict of dict,
        {'Onsager Machlup': {'gamma': 2},
        'Total Energy restraint': ... }
    File:           ~/anaconda3/envs/py37/lib/python3.7/site-packages/taps/pathfinder/dao.py
    Type:           type
    Subclasses:

We set target energy -0.45, muE 1. , tol 5e-2, gam 1. with Onsager Machlup action and energy conservation. Onsager Machlup action

$$ S_\mathrm{OM} = \frac{\Delta V}{2} + \frac{1}{4} \sum_{n=0}^{N}\left[\frac{dt}{2\gamma}\left(\left|\nabla V\left( \mathbf{x}^{\left(n+1\right)} \right)\right|^2+\left|\nabla V\left( \mathbf{x}^{\left(n\right)} \right)\right|^2\right) - \left(\nabla V\left( \mathbf{x}^{\left(n+1\right)} \right) - \nabla V \left( \mathbf{x}^{\left(n\right)} \right)\right) \cdot  \mathbf{v}^{\left(n\right)}+\frac{\gamma}{dt}\mathbf{v}^{\left(n\right)\mathbf{T}} \cdot \mathbf{v}^{\left(n\right)} \right] $$

with additional energy conservation restraint

$$\Theta_\mathrm{OM}\left(\left\{\mathbf{X}_\ast\right\},E_\mathrm{t}\right)=S_\mathrm{OM}+\mu_\mathrm{E}\sum_{n=0}^{N-1}\left(E^{\left(n\right)}-E_\mathrm{t}\right)^2$$

```python
from taps.pathfinder import DAO

action_kwargs = {
    'Onsager Machlup':{
        'gam': 1.,
    },
    'Energy Restraint':{
        'muE': 1.,
        'Et': -0.45
    }
}

search_kwargs = {"method":"L-BFGS-B"}

finder = DAO(action_kwargs=action_kwargs,
             search_kwargs=search_kwargs)

paths.finder = finder
paths.coords.epoch=6
paths.search()
```

    =================================
          Parameters
    =================================
    Onsager Machlup
      gam        : 1.0

    Energy Restraint
      muE        : 1.0
      Et         : -0.45

                Iter   nfev   njev        S   dS_max
    Converge :  11877  12169  12169   1.8258   0.0128
    Converge :  11879  12175  12175   1.8258   0.0017

    =================================
                Results
    =================================
     Onsager Machlup             : 1.2553677166473254
     Energy Restraint            : 0.5704382610974073
     Total S                     : 1.8258059777447326

```python
view(paths, viewer='MullerBrown')
```

```python

```

```python

```


```python
from numpy import array
from sympy import symbols, binomial, Sum, factorial, oo
import random
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import axes3d
from mpl_toolkits.mplot3d.art3d import Poly3DCollection  # appropriate import to draw 3d polygons
from matplotlib import style
```

# Глава 2

## Задача 2

```python
N = 20
Ks = []
for n in range(1,N):
  k = 2
  eta1 = random.random()
  eta2 = random.random()
  while eta1 > eta2:
    eta1 = eta2
    eta2 = random.random()
    k += 1
  Ks.append(k)

MK = array(Ks).mean()
MK
```

    2.473684210526316

```python
n, i = symbols('n i', positive = True,  Integer=True)
# Количество перестановок n чисел
nn = factorial(n)
# Вероятность выпадения нужной перестановки (нужная перестановка единственна)
pn = 1/nn
Sum(pn, (n, 0, oo)).doit()
```

$\displaystyle e$

## Задача 3

```python
N = 20
Ls = []
for n in range(1,N):
  l = 1
  eta = random.random()
  while eta<=1:
    eta +=random.random()
    l+=1
  Ls.append(l)

ML = array(Ls).mean()
ML
```

    2.6842105263157894

Формулы свёртки:

$F_{X+Y}(z)=P(X+Y \leq z)=\int_{-\infty}^{\infty}\int_{-\infty}^{z-y}f_X dx f_Y dy=\int_{-\infty}^\infty F_X(z-y)f_Y dy$

$f_{X+Y}(z)=\frac{d}{dz}\int_{-\infty}^\infty F_X(x-y)f_Ydy=\int_{-\infty}^\infty f_X (z-y)f_Ydy $

$P(L=n) = P(L>n-1)-P(L>n)$

Пусть есть выборка удовлетворяющая двум условиям:

$\eta_1+\eta_2+...+\eta_n>1$

$\eta_1+\eta_2+...+\eta_{n-1} \leq 1$

$P(L=2)=P(\eta_1+\eta_2>1)=1-P(\eta_1+\eta_2 \leq 1)=1-\int_0^1 \int_0^{1-y}f_X(x)dx f_Y(y)dy$

$P(L=3)=P(L \leq 3)-P(L < 3)= P(L \leq 3)-P(L = 2)$

$P(L \leq 3) = P(\eta_1+\eta_2+\eta_3>1)=1-P(\eta_1+\eta_2+\eta_3 \leq 1)=
1-\int_0^1 \int_0^{1-z} \int_0^{1-z-y}f_X(x)dx f_Y(y)dy f_Z(z)dz$

$P(L=n)=P(L \leq n)-P(L\leq n-1)$

```python
plt.figure('SPLTV',figsize=(12,6))
ax=plt.subplot(121,projection='3d')
x1=array([1, 0, 0])
y1=array([0, 1, 0])
z1=array([0, 0, 1])
ax.scatter(x1,y1,z1)

# 1. create vertices from points
verts = [list(zip(x1, y1, z1))]
# 2. create 3d polygons and specify parameters
srf = Poly3DCollection(verts, alpha=.25, facecolor='#800000')
# 3. add polygon to the figure (current axes)
plt.gca().add_collection3d(srf)

ax.view_init(30, 30)

ax.set_xlabel('x')
ax.set_ylabel('y')
ax.set_zlabel('z')
plt.show()
```

Объем n-мерного симплекса равен:

$V_n=\frac{1}{n!}$

Тогда из общей формулы получим:

$P(L=n)=P(L \leq n)-P(L\leq n-1) = (1-\frac{1}{n!}) - (1-\frac{1}{(n-1)!})= \frac{n-1}{n!}$

$ML = \sum_{n=2}^\infty n \cdot P(L=n)=\sum_{n=2}^\infty n \cdot\frac{n-1}{n!}=e$

```python
n = symbols('n', positive=True, Integer=True)
Sum(n*(n-1)/factorial(n),[n,2,oo]).doit()
```

$\displaystyle e$

## Задача 4

$M(N|X_0=x_0)=\sum_{n=0}^\infty P(N>n|X_0=x_0)$

```python

```


# What are factor models?
By Evgenia "Jenny" Nitishinskaya and Delaney Granizo-Mackenzie

Notebook released under the Creative Commons Attribution 4.0 License.

---

(Linear) factor models try to express a rate of return of an asset $i$ in terms of some factors as

$$R_i = a_i + b_{i1} F_1 + b_{i2} F_2 + \ldots + b_{iK} F_K + \epsilon_i$$

where the variables $F_j$ are called factors and the coefficients $b_{ij}$ are called factor sensitivities. The error term $\epsilon_i$ has mean zero and represents the asset-specific random fluctuation, which can be eliminated away by combining diverse assets into a portfolio. Notice that the factors are not indexed by $i$. This reflects the fact that factor models are intended to be applicable for different return streams, and are not asset-specific. Therefore we can use one model to evaluate or predict the returns on different assets and portfolios.

What are the factors that we use in the model? One possibility is the return on the market: a model with only this one factor is what we use for beta hedging. Other potential factors include inflation, changes in industrial production, and an asset's price-to-book ratio. We discuss the different kinds of factors in more detail below, but the overarching principle is that, as with regression models we have conisered in the past, we should have an economic reason to believe that the factors we choose might explain returns. We also want factors that explain the returns of many assets.

To get the model for a portfolio, we simply take the weighted average of the models for the constituent assets. If our portfolio is $P = \sum_{i=1}^n w_i x_i$, then our portfolio model is

$$ R_p = \sum_{i=1}^n w_i R_i = \sum_{i=1}^n w_i a_i + \left(\sum_{i=1}^n w_i b_{i1} \right) F_1 + \left(\sum_{i=1}^n w_i b_{i2} \right) F_2 + \ldots + \left(\sum_{i=1}^n w_i b_{iK} \right) F_K + \sum_{i=1}^n w_i \epsilon_i $$

For factor analysis to be valid, we need the error term $\epsilon_i$ to be uncorrelated across assets $i$ (so that it does in fact represent asset-specific risk), and uncorrelated with the factors $F_1,\ldots, F_k$ (if it is correlated with some factor $F_j$, then we have chosen $b_{ij}$ incorrectly, which has forced the error term to compensate).

# Types of factor models

Factor models can be categorized into three main groups, depending on the types of factors they use. There are also mixed factor models, which combine factors from different groups.

## Macroeconomic factor models

In this model, the variable $a_i$ is the expected return, and the factors are surprises (actual value minus expected value) in macroeconomic variables. That is, we have some prediction $a_i$ for the return which depends on our predictions of variables like inflation, and surprises in these variables will cause the actual returns to deviate from the expectation. So, the actual return is broken down into the expected return, unexpected return resulting from unexpected changes in the factors, and an error term. If we take the expected value of both sides, we get $a_i$, since by definition $F_j$ and $\epsilon_i$ have expected value 0:

\begin{align}
E(R_i) &= E(a_i + b_{i1} F_1 + b_{i2} F_2 + \ldots + b_{iK} F_K + \epsilon_i) \\
&= E(a_i) + E(b_{i1} F_1) + E(b_{i2} F_2) + \ldots + E(b_{iK} F_K) + E(\epsilon_i) \\
&= a_i + b_{i1} E(F_1) + b_{i2} E(F_2) + \ldots + b_{iK} E(F_K) + 0 \\
a_i &= a_i + 0 + 0 + \ldots + 0 + 0
\end{align}

We use a regression to calculate the coefficients $b_{ij}$ (one regression for each asset $i$, using historical data). The meanings of these factor sensitivities are the same as in any linear regression: we predict that every unit of surprise in variable $j$, which is an increase of $F_j$ by one unit, will cause an increase of $b_{ik}$ in the returns of stock $i$, assuming that the other factors are held constant.

## Fundamental factor models

Fundamentals are data having to do with the asset issuer, like the sector, size, and expenses of the company. In the fundamental factor model, we use the same equation as before, but interpret the terms differently. The factors $F_j$ represent the returns associated with some fundamental characteristics.

## Statistical factor models

This type of model computes the factors that best describe returns using statistical methods, without specifying what the factors represent. The two main methods for obtaining the factors are principal component analysis (resulting in factors that are the portfolios best explaining historical return variances) and factor analysis (factors are portfolios that best explain historical return covariances). After we compute the portfolios we want to use as factors, we can estimate the loadings with linear regression.

# Currently used models

There are many factor models in use in academia and industry today, whose factors were chosen based on extensive research. Among these are the BIRR model (which is macroeconomic) and the Fama-French three-factor model  (which is a mixed model, using market cap, P/B ratio, and market returns).

lectures/drafts/Theory of factor models.ipynb


```python
!pip install qiskit
from qiskit import QuantumCircuit, assemble, Aer
from qiskit.visualization import plot_histogram, plot_bloch_vector
```

    Collecting qiskit
      Downloading qiskit-0.34.2.tar.gz (13 kB)
    Collecting qiskit-terra==0.19.2
      Downloading qiskit_terra-0.19.2-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (6.5 MB)
         |████████████████████████████████| 6.5 MB 4.1 MB/s
    Collecting qiskit-aer==0.10.3
      Downloading qiskit_aer-0.10.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (18.0 MB)
         |████████████████████████████████| 18.0 MB 77 kB/s
    Collecting qiskit-ibmq-provider==0.18.3
      Downloading qiskit_ibmq_provider-0.18.3-py3-none-any.whl (238 kB)
         |████████████████████████████████| 238 kB 60.2 MB/s
    Collecting qiskit-ignis==0.7.0
      Downloading qiskit_ignis-0.7.0-py3-none-any.whl (200 kB)
         |████████████████████████████████| 200 kB 54.2 MB/s
    Requirement already satisfied: scipy>=1.0 in /usr/local/lib/python3.7/dist-packages (from qiskit-aer==0.10.3->qiskit) (1.4.1)
    Requirement already satisfied: numpy>=1.16.3 in /usr/local/lib/python3.7/dist-packages (from qiskit-aer==0.10.3->qiskit) (1.21.5)
    Requirement already satisfied: urllib3>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from qiskit-ibmq-provider==0.18.3->qiskit) (1.24.3)
    Collecting requests-ntlm>=1.1.0
      Downloading requests_ntlm-1.1.0-py2.py3-none-any.whl (5.7 kB)
    Collecting websocket-client>=1.0.1
      Downloading websocket_client-1.3.1-py3-none-any.whl (54 kB)
         |████████████████████████████████| 54 kB 2.5 MB/s
    Requirement already satisfied: python-dateutil>=2.8.0 in /usr/local/lib/python3.7/dist-packages (from qiskit-ibmq-provider==0.18.3->qiskit) (2.8.2)
    Requirement already satisfied: requests>=2.19 in /usr/local/lib/python3.7/dist-packages (from qiskit-ibmq-provider==0.18.3->qiskit) (2.23.0)
    Collecting retworkx>=0.8.0
      Downloading retworkx-0.11.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.6 MB)
         |████████████████████████████████| 1.6 MB 50.2 MB/s
    Requirement already satisfied: setuptools>=40.1.0 in /usr/local/lib/python3.7/dist-packages (from qiskit-ignis==0.7.0->qiskit) (57.4.0)
    Requirement already satisfied: dill>=0.3 in /usr/local/lib/python3.7/dist-packages (from qiskit-terra==0.19.2->qiskit) (0.3.4)
    Collecting stevedore>=3.0.0
      Downloading stevedore-3.5.0-py3-none-any.whl (49 kB)
         |████████████████████████████████| 49 kB 5.5 MB/s
    Collecting scipy>=1.0
      Downloading scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
         |████████████████████████████████| 38.1 MB 1.6 MB/s
    Collecting tweedledum<2.0,>=1.1
      Downloading tweedledum-1.1.1-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (943 kB)
         |████████████████████████████████| 943 kB 48.1 MB/s
    Requirement already satisfied: sympy>=1.3 in /usr/local/lib/python3.7/dist-packages (from qiskit-terra==0.19.2->qiskit) (1.7.1)
    Requirement already satisfied: psutil>=5 in /usr/local/lib/python3.7/dist-packages (from qiskit-terra==0.19.2->qiskit) (5.4.8)
    Collecting python-constraint>=1.4
      Downloading python-constraint-1.4.0.tar.bz2 (18 kB)
    Collecting ply>=3.10
      Downloading ply-3.11-py2.py3-none-any.whl (49 kB)
         |████████████████████████████████| 49 kB 4.1 MB/s
    Collecting symengine>=0.8
      Downloading symengine-0.9.2-cp37-cp37m-manylinux2010_x86_64.whl (37.5 MB)
         |████████████████████████████████| 37.5 MB 1.3 MB/s
    Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil>=2.8.0->qiskit-ibmq-provider==0.18.3->qiskit) (1.15.0)
    Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests>=2.19->qiskit-ibmq-provider==0.18.3->qiskit) (3.0.4)
    Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests>=2.19->qiskit-ibmq-provider==0.18.3->qiskit) (2021.10.8)
    Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests>=2.19->qiskit-ibmq-provider==0.18.3->qiskit) (2.10)
    Collecting ntlm-auth>=1.0.2
      Downloading ntlm_auth-1.5.0-py2.py3-none-any.whl (29 kB)
    Collecting cryptography>=1.3
      Downloading cryptography-36.0.2-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB)
         |████████████████████████████████| 3.6 MB 46.4 MB/s
    Requirement already satisfied: cffi>=1.12 in /usr/local/lib/python3.7/dist-packages (from cryptography>=1.3->requests-ntlm>=1.1.0->qiskit-ibmq-provider==0.18.3->qiskit) (1.15.0)
    Requirement already satisfied: pycparser in /usr/local/lib/python3.7/dist-packages (from cffi>=1.12->cryptography>=1.3->requests-ntlm>=1.1.0->qiskit-ibmq-provider==0.18.3->qiskit) (2.21)
    Requirement already satisfied: importlib-metadata>=1.7.0 in /usr/local/lib/python3.7/dist-packages (from stevedore>=3.0.0->qiskit-terra==0.19.2->qiskit) (4.11.2)
    Collecting pbr!=2.1.0,>=2.0.0
      Downloading pbr-5.8.1-py2.py3-none-any.whl (113 kB)
         |████████████████████████████████| 113 kB 57.2 MB/s
    Requirement already satisfied: typing-extensions>=3.6.4 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata>=1.7.0->stevedore>=3.0.0->qiskit-terra==0.19.2->qiskit) (3.10.0.2)
    Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata>=1.7.0->stevedore>=3.0.0->qiskit-terra==0.19.2->qiskit) (3.7.0)
    Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.7/dist-packages (from sympy>=1.3->qiskit-terra==0.19.2->qiskit) (1.2.1)
    Building wheels for collected packages: qiskit, python-constraint
      Building wheel for qiskit (setup.py) ... done
      Created wheel for qiskit: filename=qiskit-0.34.2-py3-none-any.whl size=11805 sha256=aab4114524df49b67a1a7f796a209c51f8d9b2bb90be534e71a2c8c8486d931a
      Stored in directory: /root/.cache/pip/wheels/62/77/65/cda6eedfdd2a525bd3f479a4386930ae3088a1eb01f8c944ed
      Building wheel for python-constraint (setup.py) ... done
      Created wheel for python-constraint: filename=python_constraint-1.4.0-py2.py3-none-any.whl size=24081 sha256=5d7fd3b99bddd0843e476c9f03af7f72eb60dd47e8d4bdf32d99fc5959ecc5bf
      Stored in directory: /root/.cache/pip/wheels/07/27/db/1222c80eb1e431f3d2199c12569cb1cac60f562a451fe30479
    Successfully built qiskit python-constraint
    Installing collected packages: pbr, tweedledum, symengine, stevedore, scipy, retworkx, python-constraint, ply, ntlm-auth, cryptography, websocket-client, requests-ntlm, qiskit-terra, qiskit-ignis, qiskit-ibmq-provider, qiskit-aer, qiskit
      Attempting uninstall: scipy
        Found existing installation: scipy 1.4.1
        Uninstalling scipy-1.4.1:
          Successfully uninstalled scipy-1.4.1
    ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
    albumentations 0.1.12 requires imgaug<0.2.7,>=0.2.5, but you have imgaug 0.2.9 which is incompatible.
    Successfully installed cryptography-36.0.2 ntlm-auth-1.5.0 pbr-5.8.1 ply-3.11 python-constraint-1.4.0 qiskit-0.34.2 qiskit-aer-0.10.3 qiskit-ibmq-provider-0.18.3 qiskit-ignis-0.7.0 qiskit-terra-0.19.2 requests-ntlm-1.1.0 retworkx-0.11.0 scipy-1.7.3 stevedore-3.5.0 symengine-0.9.2 tweedledum-1.1.1 websocket-client-1.3.1

```python
from math import sqrt, pi
```

```python
qc = QuantumCircuit(1)
```

```python
initial_state = [0,1] ## Definig Initial State as |1>
qc.initialize(initial_state,0) ## Apply the initialization operation on the 0th qubit
qc.draw()
```

   ┌─────────────────┐
q: ┤ Initialize(0,1) ├
   └─────────────────┘

```python
from qiskit import execute

result1 = execute(qc, Aer.get_backend('aer_simulator')).result()

```

```python
qc.save_statevector()
output_state = result1.get_statevector()
```

```python
print(output_state)
```

    Statevector([0.+0.j, 1.+0.j],
                dims=(2,))

```python
qc.measure_all()
qc.draw()
```

        ┌─────────────────┐ ░  ░  ░ ┌─┐
     q: ┤ Initialize(0,1) ├─░──░──░─┤M├
        └─────────────────┘ ░  ░  ░ └╥┘
meas: 1/═════════════════════════════╩═
                                     0

```python
counts = execute(qc, Aer.get_backend('qasm_simulator')).result().get_counts()
##plot_histogram(counts)
```

    Simulation failed and returned the following error message:
    ERROR: Failed to load qobj: Duplicate key "statevector" in save instruction.

```python
initial_state = [1/sqrt(2), 1j/sqrt(2)]
qc1 = QuantumCircuit(1) # Must redefine qc
qc1.initialize(initial_state, 0) # Initialize the 0th qubit in the state `initial_state`
qc1.save_statevector() # Save statevector
sim = Aer.get_backend('aer_simulator')
qobj = assemble(qc1)
result = sim.run(qobj).result()
state = sim.run(qobj).result().get_statevector() # Execute the circuit
print(state)
```

    Statevector([0.70710678+0.j        , 0.        +0.70710678j],
                dims=(2,))

```python
qobj = assemble(qc1)
results = sim.run(qobj).result().get_counts()
plot_histogram(results)
```

```python

```


# Transformadores
## Ejercicios para practicar
__U.N.L.Z. - Facultad de Ingeniería__
__Electrotecnia__
__Alumno:__ Daniel Antonio Lorenzo

## Problema N°1
Un transformador monofásico de 380/110V cuya $S_N = 5 \, \mathrm{kVA}$. Ensayado con el devanado de baja tensión (o secundario) en cortocircuito; se debe aplicar 33V al lado primario para que circule corriente nominal, siendo la potencia consumida por la máquina $P_{CC} = 100,2 \, \mathrm{W}$.

1. Cuál será la regulación, cuando la carga aplicada tiene un factor de potencia del 0,8 en atraso, y consume la corriente nominal.
2. Y el rendimiento, cuando la carga aplicada tiene un factor de potencia del 0,5 en atraso, entregando su corriente nominal, y las pérdidas en el núcleo son de 80W.

### Solución

__ítem a)__

__Datos:__
$\left\{
\begin{array}{l}
S_N = 5000 \, \mathrm{VA} \\
U_{1N} = 380 \, \mathrm{V} \\
U_{2N} = 110 \, \mathrm{V} \\
U_{CC} = 33 \, \mathrm{V} \\
P_{CC} = 100,2 \, \mathrm{W} \\
\mathrm{fp} = 0,8
\end{array}
\right.$

__Cálculo de corriente nominal 1__
$$ I_{1N} = \frac{S_N}{U_{1N}} = 13,16 \, \mathrm{A} $$

__Cálculo de corriente nominal 2__
$$ I_{2N} = \frac{S_N}{U_{2N}} = 45.45 \, \mathrm{A} $$

__Cálculo de $\varphi_e$__
$$ P_{CC} = U_{CC} I_{1N} \cos \varphi_e $$

$$ \varphi_e = \frac{P_{CC}}{U_{CC} I_{1N}} = 78,7^\circ $$

__Cálculo de $\varphi_2$__
$$ \cos \varphi_2 = \mathrm{fp} = 0,8 $$

$$ \varphi_2 = \arccos 0,8 = 36,87^\circ $$

__Fasorial:__

Del diagrama fasorial se deduce:
$$ U_1^2 = [U'_2 + U'_e \cos (\varphi_{e} - \varphi_2)]^2 + [U'_e \sin (\varphi_e - \varphi_2)]^2 $$

Cálculo auxiliar

$\left\{
\begin{array}{l}
U'_e \cos (\varphi_e - \varphi_2) = 33 \cos (78,7 - 36,9) = 24,7 \, \mathrm{V} \\
U'_e \sin (\varphi_e - \varphi_2) = 33 \sin (78,7 - 36,9) = 21,9 \, \mathrm{V}
\end{array}
\right.$

Entonces

$$ 380^2 = (U'_2 + 24,7)^2 + (21,9)^2 $$

$$ (U'_2)^2 + 49,4 U'_2 - 143310 = 0 $$

$$ U'_2 = 354,7 \, \mathrm{V} $$

__El porcentaje de regulación resulta:__

$$ \varepsilon \% = \frac{U_1 - U'_2}{U_1} \times 100 $$

$$ \varepsilon \% = \frac{380 - 354,7}{380} \times 100 = 6,7\% $$

__ítem b)__

__Datos:__
$\left\{
\begin{array}{l}
\mathrm{fp}_b = 0,5 \\
P_0 = 80 \, \mathrm{W}
\end{array}
\right.$

$$I_2 = I'_2 \cdot a = 45,45 \, \mathrm{V}$$

$$ U_2 = \frac{U'_2}{a} = 102,65 \, \mathrm{V} $$

$$ \varphi_b = \arccos (\mathrm{fp}_b) = 60^\circ$$

$$ \eta = \frac{U_2 I_2 \cos \varphi_b}{U_2 I_2 \cos \varphi_b + P_{cc} + P_0} \times 100 = 92,8 \% $$

(Resolución en python3)

```python
import numpy as np
```

```python
# Datos:
SN  =  5e3  # VA
U1N = 380   # V
U2N = 110   # V
Ucc =  33   # V
Pcc = 100.2 # W
fp  = 0.8
```

```python
# Corriente nominal primaria
I1N = SN/U1N
# Corriente nominal secundaria
I2N = SN/U2N
# (relación de transformación)
a = U1N/U2N
```

```python
phi_cc = np.arccos( Pcc/(Ucc*I1N) )
phi_2 = np.arccos(fp)
```

```python
print('phi_cc = %.2f°'%np.rad2deg(phi_cc))
print('phi_2 = %.2f°'%np.rad2deg(phi_2))
```

    phi_cc = 76.66°
    phi_2 = 36.87°

```python
import sympy as sp
```

```python
U2eq = sp.symbols('U2')
Eq = sp.Eq(U1N**2 , (U2eq + Ucc*sp.cos(phi_cc - phi_2))**2 + (Ucc*sp.sin(phi_cc - phi_2)**2) )
```

```python
Sol = sp.solve(Eq,U2eq)
```

```python
U2eq = Sol[1]
```

```python
print('U2eq = %.2f V'%U2eq)
```

    U2eq = 354.62 V

```python
reg = (U1N - U2eq)/U1N * 100
```

```python
print('reg = %.2f %%'%reg)
```

    reg = 6.68 %

```python
##########
# Item b
##########
I2eq = I1N
I2 = I2eq*a
U2 = U2eq/a
fp_b = 0.5
phi_b = np.arccos(fp_b)
Po = 80 # W

n = (U2*I2*np.cos(phi_b))/(U2*I2*np.cos(phi_b) + Po + Pcc) * 100
```

```python
print('-------------------')
print('Rendimiento:')
print('n = %.1f %%'%n)
print('-------------------')
```

    -------------------
    Rendimiento:
    n = 92.8 %
    -------------------

```python
%reset -s -f
```

## Problema N°2
Se desea conocer el valor más alto del rendimiento que puede alcanzar un transformador monofásico, con los siguientes valores nominales: $S_N = 14 \, \mathrm{kVA}$, $800/400 \, \mathrm{V}$, y conociendo que: $U_{CC} = 3 \, \%$, $P_{fe} = 95 \, \mathrm{W}$ y $P_{CC} = 180 \, \mathrm{W}$.

### Solución

__Datos:__
$\left\{
\begin{array}{l}
S_N = 14 \, \mathrm{kVA} \\
U_{N1} = 800 \, \mathrm{V} \\
U_{N2} = 400 \, \mathrm{V} \\
U_{cc} = 3 \% \\
P_{fe} = 95 \, \mathrm{W} \\
P_{cc} = 180 \, \mathrm{W}
\end{array}
\right.$

__Relación de transformación__
$$ a = \frac{U_{N1}}{U_{N2}} = 2 $$

__Corriente nominal primaria__
$$ I_{N1} = \frac{S_N}{U_{N1}} = 17,5 \, A $$

__Corriente nominal secundaria__
$$  I_{N2} = \frac{S_N}{U_{N2}} = 35 \, A $$

__Vacío:__

En el ensayo de vacío __se aplica la tensión nominal en el lado de baja tensión__ del transformador mientras que __el lado de alta tensión queda en circuito abierto.__

$$U_0 = U_{N2} = 400 \, \mathrm{V}$$

$P_0 = P_{fe}$
$$ I_0 = \frac{P_0}{U_0} = 0,2375 \, A $$

$$ P_0 = I_0 U_0 \cos \varphi_0 $$

Entonces
$$ \varphi_0 = \arccos \left( \frac{P_0}{I_0 U_0} \right) $$

__Cortocircuito:__

$$ I_{cc} = I_{N2} $$

$$ Z_{eq1} = \frac{U_{cc}}{I_{cc}} $$

$$ \varphi_{cc} = \arccos \left( \frac{P_{cc}}{I_{cc} U_{cc}} \right) = 64,62^\circ $$

$$ R_{eq1} = Z_{eq1} \cos \varphi_{cc} \quad \Rightarrow \quad R_{eq2} = \frac{R_{eq1}}{a^2} $$

$$ X_{eq1} = Z_{eq1} \sin \varphi_{cc} \quad \Rightarrow \quad X_{eq2} = \frac{X_{eq1}}{a^2} $$

__Rendimiento máximo:__

$$ P_0 = I_2^2 R_{eq2} \quad \Rightarrow \quad I_\mathrm{2 \eta max} = \sqrt{\frac{P_0}{R_{eq2}}} $$

$U_{20} = U_0$

$$ U_\mathrm{2 \eta max} = U_{20} - I_\mathrm{2 \eta max} (R_{eq2} \cdot 1 + 0) $$

$$ \eta_\mathrm{max} = \frac{U_\mathrm{2 \eta max} I_\mathrm{2 \eta max} \cdot 1}{U_\mathrm{2 \eta max} I_\mathrm{2 \eta max} \cdot 1 + P_{cu} + P_0} \times 100 = 98,7 \% $$

(Resolución en python3)

```python
import numpy as np
```

```python
# Datos:
SN  = 14000     # VA
UN1 =   800     # V
UN2 =   400     # V
Po  =    95     # W
Pcc =   180     # W
Ucc = 3/100*UN1
```

```python
a = UN1/UN2  #     (relación de transformación)
IN1 = SN/UN1 # A   (corriente nominal primaria)
IN2 = SN/UN2 # A   (corriente nominal secundaria)
```

```python
# Vacío
Uo = UN2
Io = Po/Uo

phi_o = np.arccos(Po/(Io*Uo))

Ip = Io*np.cos(phi_o)
Im = Io*np.sin(phi_o)

# Corto circuito
Icc = IN2
phi_cc = np.arccos(Pcc/(Icc*Ucc))

Zeq1 = Ucc/Icc
Req1 = Zeq1*np.cos(phi_cc)
Xeq1 = Zeq1*np.sin(phi_cc)
Req2 = Req1/a**2
Xeq2 = Xeq1/a**2

# Rendimiento
U20 = Uo
I2max = np.sqrt(Po/Req2)
U2max = U20 - I2max*(Req2*1 + 0)

n = (U2max*I2max*1)/(U2max*I2max + Pcc + Po) * 100
```

```python
print('-------------------')
print('Rendimiento máximo:')
print('n = %.1f %%'%n)
print('-------------------')
```

    -------------------
    Rendimiento máximo:
    n = 98.7 %
    -------------------

--------------------