# Pricing with Rational Inattention: Partial Equilibrium with Endogenous Feedback

This notebook goes through a couple of examples for solving LQG Dynamic Rational Inattention models using the methods and the [solver](https://github.com/choongryulyang/dynamic_multivariate_RI) from [Afrouzi and Yang (2019)](http://www.afrouzi.com/dynamic_inattention.pdf) that was adapted for Julia by [Miguel Acosta](https://www.acostamiguel.com/home).

Include the solver and import packages for plots and performance:

```julia
include("DRIP.jl");
using Plots, LaTeXStrings, BenchmarkTools, Suppressor; pyplot();
```

## Example 2: Pricing with Endogenous Feedback (MA Approximation)
### Background
Suppose now that there is general equilibirum feedback with the degree of strategic complementarity $\alpha$: $$p_{i,t}^*=(1-\alpha)q_t+\alpha p_t$$ where
\begin{align}
    \Delta q_t&=\rho \Delta q_{t-1}+u_t,\quad u_t\sim \mathcal{N}(0,\sigma_u^2) \\
    p_t&\equiv \int_0^1 p_{i,t}di
\end{align}
Note that now the state space representation for $p_{i,t}^*$ is no longer exogenous and is determined in the equilibrium. However, we know that this is a Guassian process and by Wold's theorem we can decompose it to its $MA(\infty)$ representation:
\begin{align}
    p_{i,t}^*=\Phi(L)u_t
\end{align}
where $\Phi(.)$ is a lag polynomial and $u_t$ is the shock to nominal demand. Here, we have basically guessed that the process for $p_{i,t}^*$ is determined uniquely by the history of monetary shocks which requires that rational inattention errors of firms are orthogonal (See [Afrouzi (2020)](www.afrouzi.com/strategic_inattetion.pdf)). Our objective is to find $\Phi(.)$.

We cannot put $MA(\infty)$ processes in the computer and have to truncate them. However, we know that for stationary processes we can arbitrarily get close to the true process by truncating $MA(\infty)$ processes. Our problem here is that $p_{i,t}^*$ has a unit root and is not stationary. We can bypass this issue by re-writing the state space in the following way:
\begin{align}
    p_{i,t}^*=\phi(L)\tilde{u}_t,\quad \tilde{u}_t=(1-L)^{-1}u_t =\sum_{j=0}^\infty u_{t-j}
\end{align}
here $\tilde{u}_{t-j}$ is the unit root of the process and basically we have differenced out the unit root from the lag polynomial, and $\phi(L)=(1-L)\Phi(L)$. Notice that since the original process was difference stationary, differencing out the unit root means that $\phi(L)$ is now in $\ell_2$, and the process can now be approximated arbitrarily precisely with truncation.

### Matrix Notation

For a length of truncation $L$, let $\vec{x}_t\equiv (\tilde{u}_t,\tilde{u}_{t-1},\dots,\tilde{u}_{t-(L+1)})\in\mathbb{R}^L$. Then, note that $p_{i,t}^*\approx \mathbf{H} '\vec{x}_{t}$ where $\mathbf{H}\in \mathbb{R}^L$ is the truncated matrix analog of the lag polynominal, and is endogenous to the problem. Our objective is to find the general equilibrium $\mathbf{H}$ along with the optimal information structure that it implies.

Moreover, note that $$q_t=\mathbf{H}_q'\vec{x}_t,\quad \mathbf{H}_q'=(1,\rho,\rho^2,\dots,\rho^{L-1})$$

We will solve for $\phi$ by iterating over the problem. In particular, in iteration $n\geq 1$, given the guess $\mathbf{H}_{(n-1)}$, we have the following state space representation for the firm's problem

\begin{align}
   \vec{x}_{t}&
    = \underset{\mathbf{A}}{\underbrace{\left[\begin{array}{ccccc}
        1 & 0 & \dots & 0 & 0\\
        1 & 0 & \dots & 0 & 0\\
        0 & 1 & \dots & 0 & 0\\
        \vdots & \vdots & \ddots & \vdots & \vdots\\
        0 & 0 & \dots & 1 & 0
        \end{array}\right]}}\, \vec{x}_{t-1}
    + \underset{\mathbf{Q}}{\underbrace{\left[\begin{array}{c}
        \sigma_u\\
        0\\
        0\\
        \vdots\\
        0
        \end{array}\right]}}\, u_t, \\
        p_{i,t}^*&=\mathbf{H}_{(n-1)}'\vec{x}_{t}
\end{align}

Then we can solve the ratinoal inattention problem of all firms and get the new guess for $p_t^*$:
\begin{align}
    p_t^* & = (1-\alpha)q_t + \alpha p_t \\
      & = (1-\alpha)\sum_{j=0}^\infty \alpha^j q_t^{(j)},  \\
      q_{t}^{(j)}&\equiv \begin{cases}
q_{t} & j=0\\
\int_{0}^{1}\mathbb{E}_{i,t}[q_{t}^{(j-1)}]di & j\geq1
\end{cases}
\end{align}

where $q_t^{(j)}$ is the $j$'th order belief of firms, on average, of $q_t$. Now, we need to write these higher order beliefs in terms of the state vector. Suppose, for a given $j$, there exists $\mathbf{X}_j\in \mathbb{R}^{L\times L}$ such that
$$ q_t^{(j)} = \mathbf{H}_q'\mathbf{X}_j \vec{x}_t $$
This clearly holds for $j=0$ with $\mathbf{X}_0=\mathbf{I}$.

Now, note that

\begin{align}
    q_{t}^{(j+1)} &= \int_{0}^{1}\mathbb{E}_{i,t}[q_{t}^{(j)}]di \\
                  &= \mathbf{H}_{q}'\mathbf{X}_{j}\int_{0}^{1}\mathbb{E}_{i,t}[\vec{x}_{t}]di \\
                  &= \mathbf{H}_{q}'\mathbf{X}_{j}\sum_{j=0}^{\infty}[(\mathbf{I}-\mathbf{K}_{(n)}\mathbf{Y}'_{(n)})\mathbf{A}]^{j}\mathbf{K}_{(n)}\mathbf{Y}'_{(n)}\vec{x}_{t-j} \\
                  &\approx\underset{\equiv\mathbf{X}_{(n)}}{\mathbf{H}_{q}'\mathbf{X}_{j}\underbrace{\left[\sum_{j=0}^{\infty}[(\mathbf{I}-\mathbf{K}_{(n)}\mathbf{Y}'_{(n)})\mathbf{A}]^{j}\mathbf{K}_{(n)}\mathbf{Y}'_{(n)}\mathbf{M}'^{j}\right]}}\vec{x}_{t}=\mathbf{H}_{q}'\mathbf{X}_{j}\mathbf{X}_{(n)}\vec{x}_{t}
\end{align}

where the $(n)$ subscripts refer to the solution of the RI problem in the $(n)$'th iteration. Note that this implies
$$\mathbf{X}_{j}=\mathbf{X}_{(n)}^j,\forall j\geq 0 \Rightarrow q_t^{(j)}=\mathbf{X}_{(n)}^{j}\vec{x}_t $$

This gives us an updated guess for $\mathbf{H}$:
\begin{align}
    p_t^*&=(1-\alpha)\mathbf{H}_q'\underset{\equiv \mathbf{X}_{p,(n)}}{\underbrace{\left[\sum_{j=0}^\infty \alpha^j \mathbf{X}_{(n)}^j\right]}} \vec{x}_t \\
        &\Rightarrow \mathbf{H}_{(n)} =  (1-\alpha)\mathbf{X}_{p,(n)}'\mathbf{H}_q
\end{align}
We iterate until convergence of $\mathbf{H}_{(n)}$.

### Setting up the problem

```julia
ρ   = 0.6;        #persistence of money growth
σ_u = 0.1;        #std. deviation of shocks to money growth
α   = 0.8;        #degree of strategic complementarity
L   = 40;         #length of truncation
Hq  = ρ.^(0:L-1); #state-space rep. of Δq

## specifying the primitives of the drip
ω   = 0.2;
β   = 0.99;
A   = [1 zeros(1,L-2) 0; Matrix(I,L-1,L-1) zeros(L-1,1)];
Q   = [σ_u; zeros(L-1,1)];
```

Let us now define a function that solves the GE problem and returns the solution in a `drip` structure:

```julia
function ge_drip(ω,β,A,Q,          #primitives of drip except for H because H is endogenous
                 α,                #strategic complementarity
                 Hq,               #state space rep. of Δq
                 L;                #length of truncation
                 H0       = Hq,    #optional: initial guess for H (Hq is the true solution when α=0)
                 maxit    = 200,   #optional: max number of iterations for GE code
                 tol      = 1e-4)  #optional: tolerance for iterations
    # set primitives
    err   = 1;
    iter  = 0;
    M     = [zeros(1,L-1) 0; Matrix(I,L-1,L-1) zeros(L-1,1)];
    # iterate on GE
    while (err > tol) & (iter < maxit)
            if iter == 0
                global ge  = solve_drip(ω,β,A,Q,H0, w = 0.9);
            else
                global ge  = solve_drip(ω,β,A,Q,H0;Ω0 = ge.Ω ,Σ0 = ge.Σ_1,maxit=15);
            end

            XFUN(jj) = ((I-ge.K*ge.Y')*ge.A)^jj * (ge.K*ge.Y') * (M')^jj
            X = infinitesum(XFUN; maxit=L, start = 0);  #E[x⃗]=X×x⃗

            XpFUN(jj) = α^jj * X^(jj)
            Xp = infinitesum(XpFUN; maxit=L, start = 0);

            H1 = (1-α)*Xp'*Hq;
            err= 0.5*norm(H1-H0,2)/norm(H0)+0.5*err;
            H0 = H1;

            iter += 1;
            if iter == maxit
                print("GE loop hit maxit\n")
            end
            println("Iteration $iter. Difference: $err")
    end
    return(ge)
end;
```

Solve and measure performance for random parameters:

```julia
@suppress @benchmark ge_drip(ω,β,A,Q,α,Hq,L) setup = (ω=rand()) # solves and times the fixedp point for different values of ω
```

    BenchmarkTools.Trial:
      memory estimate:  357.18 MiB
      allocs estimate:  44505
      --------------
      minimum time:     420.432 ms (9.15% GC)
      median time:      438.778 ms (9.15% GC)
      mean time:        441.804 ms (9.09% GC)
      maximum time:     462.380 ms (9.54% GC)
      --------------
      samples:          12
      evals/sample:     1

Solve for benchmark parameterization:

```julia
@time @suppress  ge = ge_drip(ω,β,A,Q,α,Hq,L); # remove suppress to see convergence log
```

      0.466833 seconds (47.55 k allocations: 368.915 MiB, 8.77% gc time)

Get IRFs:

```julia
T      = L;
geirfs = dripirfs(ge,T);

M  = [zeros(1,T-1) 0; Matrix(I,T-1,T-1) zeros(T-1,1)];
dq = diagm(Hq)*geirfs.x[1,1,:];
Pi = (I-M)*geirfs.a[1,1,:];
y  = inv(I-M)*(dq-Pi);

p1 = plot(1:T,[dq,Pi],
     label = [L"Agg. Demand Growth ($\Delta q$)" L"Inflation ($\pi$)"]);

p2 = plot(1:T,y,
     label  = L"Output ($y$)");

plot(p1,p2,
    layout     = (1,2),
    xlim       = (1,20),
    lw         = 3,
    legend     = :topright,
    legendfont = font(12),
    tickfont   = font(12),
    size       = (900,370),
    framestyle = :box)
```


# Collaborative Filtering

- Collaborative filtering is a straightforward approach to **recommender systems**. In a nutshell it gets recommendations by copying from the most similar entities:

https://towardsdatascience.com/various-implementations-of-collaborative-filtering-100385c6dfe0

## Cosine similarity
- Cosine Similarity used to determine similarity of movie ratings, 0 being totally dissimilar, 1 being the exact same ratings

As an Euclidean similarity measure, cosine similarity is commonly used:

\begin{align}
cos(X, Y) = \frac{X*Y}{\left||X |\right| * \left||Y |\right|}
\end{align}

Or, in more detail:

\begin{align}
cos(X, Y) = \frac{\sum_i^n{X_i*Y_i}}{\sqrt{\sum_i{X_i^2}}*\sqrt{\sum_i{Y_i^2}}}
\end{align}

## Read your movie data

```python
import pandas as pd
import numpy as np
import math
import seaborn as sns
import matplotlib.pyplot as plt
%matplotlib inline
```

```python
df = pd.read_csv('peppermint_movies - Sheet1.csv')
movies = df.set_index(['Reviewer', 'Movie'])['Rating'].unstack()
movies.fillna(0.0, inplace=True)
movies # Now we have a nice movie matrix!
```

      Movie
      A Star Is Born
      Argo
      Bohemian Rhapsody
      Das Leben der Anderen
      Dirty Dancing
      Dora the Explorer
      Guesthouse Paradiso
      Harry Potter and the Sorcerer's Stone
      High School Musical 4
      IT
      ...
      Power Rangers
      Princess Diaries
      Roma
      Schindlers List
      Shawshank Redemption
      Titanic
      Toxic Avenger
      Trainspotting
      Who am I
      Zootopia

      Reviewer

      Anders
      0.0
      0.0
      0.0
      0.0
      5.0
      0.0
      0.0
      0.0
      5.0
      0.0
      ...
      0.0
      5.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0

      Josh
      4.0
      3.0
      0.0
      0.0
      2.0
      0.0
      0.0
      2.0
      0.0
      0.0
      ...
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      2.0

      Karl
      0.0
      5.0
      4.0
      4.0
      1.0
      0.0
      0.0
      2.0
      0.0
      0.0
      ...
      1.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      5.0
      0.0
      0.0

      Max
      0.0
      0.0
      0.0
      0.0
      0.0
      5.0
      1.0
      0.0
      0.0
      0.0
      ...
      5.0
      4.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0

      Michael
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      5.0
      0.0
      0.0
      0.0
      ...
      0.0
      0.0
      0.0
      0.0
      0.0
      1.0
      2.0
      0.0
      5.0
      0.0

      Nedra
      0.0
      0.0
      5.0
      4.0
      0.0
      0.0
      0.0
      0.0
      0.0
      2.0
      ...
      0.0
      0.0
      0.0
      5.0
      0.0
      2.0
      0.0
      4.0
      0.0
      0.0

      Paul
      4.0
      5.0
      5.0
      5.0
      0.0
      2.0
      0.0
      4.0
      2.0
      4.0
      ...
      0.0
      0.0
      1.0
      0.0
      5.0
      5.0
      2.0
      4.0
      0.0
      0.0

7 rows × 22 columns

## Implement a Cosine Similarity function

```python
karl = movies.loc['Karl']
paul = movies.loc['Paul']

np.dot(karl,paul)
```

    93.0

#### We have calculated the dot product of the sum of all the film ratings - for Karl and Paul

```python
def cosim(x, y):
    '''Returns the cosine similarity of two vectors'''
    num = np.dot(x,y) #numerator

    xsum = math.sqrt(sum(x ** 2))
    ysum = math.sqrt(sum(y ** 2))
    denom = xsum * ysum
    return num / denom
```

```python
# from the example done on the board
x = np.array([3, 2, 0, 1])
y = np.array([5, 4, 2, 0])
cosim(x, y)
```

    0.9163419338230351

```python
# Here we would expect 1 as they are the same! So can make an assertion
# to check that your function is correct
assert cosim(x, x) == 1.0
print(cosim(x, x))
```

    1.0

## Calculate Cosine Similarity of two people

```python
karl = movies.loc['Karl']
paul = movies.loc['Paul']

cosim(karl, paul)
```

    0.697534877615843

## Calculate an all vs. all in what we call a similarity matrix
- Pattern to calculate matrices is to do a **for loop within a for loop (NESTED)**

```python
movies
```

      Movie
      A Star Is Born
      Argo
      Bohemian Rhapsody
      Das Leben der Anderen
      Dirty Dancing
      Dora the Explorer
      Guesthouse Paradiso
      Harry Potter and the Sorcerer's Stone
      High School Musical 4
      IT
      ...
      Power Rangers
      Princess Diaries
      Roma
      Schindlers List
      Shawshank Redemption
      Titanic
      Toxic Avenger
      Trainspotting
      Who am I
      Zootopia

      Reviewer

      Anders
      0.0
      0.0
      0.0
      0.0
      5.0
      0.0
      0.0
      0.0
      5.0
      0.0
      ...
      0.0
      5.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0

      Josh
      4.0
      3.0
      0.0
      0.0
      2.0
      0.0
      0.0
      2.0
      0.0
      0.0
      ...
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      2.0

      Karl
      0.0
      5.0
      4.0
      4.0
      1.0
      0.0
      0.0
      2.0
      0.0
      0.0
      ...
      1.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      5.0
      0.0
      0.0

      Max
      0.0
      0.0
      0.0
      0.0
      0.0
      5.0
      1.0
      0.0
      0.0
      0.0
      ...
      5.0
      4.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0

      Michael
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      5.0
      0.0
      0.0
      0.0
      ...
      0.0
      0.0
      0.0
      0.0
      0.0
      1.0
      2.0
      0.0
      5.0
      0.0

      Nedra
      0.0
      0.0
      5.0
      4.0
      0.0
      0.0
      0.0
      0.0
      0.0
      2.0
      ...
      0.0
      0.0
      0.0
      5.0
      0.0
      2.0
      0.0
      4.0
      0.0
      0.0

      Paul
      4.0
      5.0
      5.0
      5.0
      0.0
      2.0
      0.0
      4.0
      2.0
      4.0
      ...
      0.0
      0.0
      1.0
      0.0
      5.0
      5.0
      2.0
      4.0
      0.0
      0.0

7 rows × 22 columns

```python
matrix = []
for name1 in movies.index:
    row = []
    for name2 in movies.index:
        x = movies.loc[name1]
        y = movies.loc[name2]
        row.append(cosim(x, y))
    matrix.append(row)

similarity_matrix = pd.DataFrame(matrix, index=movies.index,
                                 columns=movies.index)
similarity_matrix
```

      Reviewer
      Anders
      Josh
      Karl
      Max
      Michael
      Nedra
      Paul

      Reviewer

      Anders
      1.000000
      0.137038
      0.061546
      0.282138
      0.000000
      0.000000
      0.081244

      Josh
      0.137038
      1.000000
      0.265674
      0.000000
      0.000000
      0.000000
      0.325656

      Karl
      0.061546
      0.265674
      1.000000
      0.065117
      0.000000
      0.629253
      0.697535

      Max
      0.282138
      0.000000
      0.065117
      1.000000
      0.082367
      0.000000
      0.085958

      Michael
      0.000000
      0.000000
      0.000000
      0.082367
      1.000000
      0.028427
      0.085386

      Nedra
      0.000000
      0.000000
      0.629253
      0.000000
      0.028427
      1.000000
      0.585909

      Paul
      0.081244
      0.325656
      0.697535
      0.085958
      0.085386
      0.585909
      1.000000

## Plot a heatmap

```python
plt.figure(figsize=(12,8))
sns.heatmap(similarity_matrix, annot=True, cmap='Blues')
```

## Query for a new user

```python
movies.columns
```

    Index(['A Star Is Born', 'Argo', 'Bohemian Rhapsody', 'Das Leben der Anderen',
           'Dirty Dancing', 'Dora the Explorer', 'Guesthouse Paradiso',
           'Harry Potter and the Sorcerer's Stone', 'High School Musical 4', 'IT',
           'Inside Out', 'Murder on the Orient Express', 'Power Rangers',
           'Princess Diaries', 'Roma', 'Schindlers List', 'Shawshank Redemption',
           'Titanic', 'Toxic Avenger', 'Trainspotting', 'Who am I', 'Zootopia'],
          dtype='object', name='Movie')

```python
my_movies = {'A Star Is Born': 2.5,
             'Bohemian Rhapsody': 4.0,
             'Dora the Explorer': 1.0,
             'Schindlers List': 5.0
            }
```

```python
# build a list with all movies, filling up with zeroes
daniela = []
for m in movies.columns:
    if m in my_movies:
        rating = my_movies[m]
    else:
        rating = 0.0
    daniela.append(rating)

daniela = np.array(daniela)
daniela
```

    array([2.5, 0. , 4. , 0. , 0. , 1. , 0. , 0. , 0. , 0. , 0. , 0. , 0. ,
           0. , 0. , 5. , 0. , 0. , 0. , 0. , 0. , 0. ])

```python
# find most similar user
for user in movies.index:
    value = cosim(daniela, movies.loc[user])
    print(f'{user:10} {value:10.3f}')
```

    Anders          0.000
    Josh            0.171
    Karl            0.246
    Max             0.088
    Michael         0.000
    Nedra           0.683
    Paul            0.324

```python
# print top movies of that user
movies.loc['Nedra'].sort_values(ascending=False).head()
```

    Movie
    Bohemian Rhapsody        5.0
    Schindlers List          5.0
    Trainspotting            4.0
    Das Leben der Anderen    4.0
    Titanic                  2.0
    Name: Nedra, dtype: float64

### Advantages of Collaborative Filtering

* fast
* works for huge datasets
* works with other types of features (genres, demography)
* application: similar users in a social network (user-based CF)
* variation: item-based collaborative filtering --> e-commerce
* *"people who watched X also watched Y"*

## Same thing in Scikit

```python
from sklearn.metrics.pairwise import cosine_similarity
```

```python
cosine_similarity(movies)
```

    array([[1.        , 0.13703774, 0.06154575, 0.28213825, 0.        ,
            0.        , 0.08124445],
           [0.13703774, 1.        , 0.26567383, 0.        , 0.        ,
            0.        , 0.3256565 ],
           [0.06154575, 0.26567383, 1.        , 0.06511653, 0.        ,
            0.6292532 , 0.69753488],
           [0.28213825, 0.        , 0.06511653, 1.        , 0.08236662,
            0.        , 0.08595812],
           [0.        , 0.        , 0.        , 0.08236662, 1.        ,
            0.02842676, 0.08538577],
           [0.        , 0.        , 0.6292532 , 0.        , 0.02842676,
            1.        , 0.58590898],
           [0.08124445, 0.3256565 , 0.69753488, 0.08595812, 0.08538577,
            0.58590898, 1.        ]])

```python
sim_matrix = pd.DataFrame(cosine_similarity(movies), index=movies.index, columns=movies.index)
sim_matrix
```

      Reviewer
      Anders
      Josh
      Karl
      Max
      Michael
      Nedra
      Paul

      Reviewer

      Anders
      1.000000
      0.137038
      0.061546
      0.282138
      0.000000
      0.000000
      0.081244

      Josh
      0.137038
      1.000000
      0.265674
      0.000000
      0.000000
      0.000000
      0.325656

      Karl
      0.061546
      0.265674
      1.000000
      0.065117
      0.000000
      0.629253
      0.697535

      Max
      0.282138
      0.000000
      0.065117
      1.000000
      0.082367
      0.000000
      0.085958

      Michael
      0.000000
      0.000000
      0.000000
      0.082367
      1.000000
      0.028427
      0.085386

      Nedra
      0.000000
      0.000000
      0.629253
      0.000000
      0.028427
      1.000000
      0.585909

      Paul
      0.081244
      0.325656
      0.697535
      0.085958
      0.085386
      0.585909
      1.000000

```python
plt.figure(figsize=(12,8))
sns.heatmap(sim_matrix, annot=True, cmap='Blues')
```

## See also

* Tanimoto Coefficient
* Jaccard Index

### Tanimoto coefficient
For binary variables you might use the Tanimoto coefficient (also see Jaccard Index). It uses the number of bits:

\begin{align}
cos(X, Y) = \frac{nbits(X \cap Y)}{nbits(X) * nbits(Y) - nbits(X \cap Y)}
\end{align}

### User-based Collaborative Filtering
- Find the N most similar users. Aggregate their ratings on the items to be ranked.

### Item-based Collaborative Filtering

- “Users who bought X also bought Y”

Create a similarity matrix of items and find the N most similar items to those rated by an user.

```python

```


# **Chemostat: Logistic + Clogging**
by: Edwin Saavedra C.

```python
#%matplotlib widget
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.gridspec as gridspec
import ipywidgets as wd
import sympy as sp
import pandas as pd
from IPython.display import display
from scipy.integrate import odeint
from scipy.optimize import fsolve,root
```

```python
## MATPLOTLIB rcParams
plt.rcParams['axes.labelsize'] = 14
plt.rcParams['axes.labelweight'] = 'medium'
plt.rcParams['axes.labelpad'] = 5.0
plt.rcParams['legend.fontsize'] = 10
plt.rcParams['axes.edgecolor'] = 'gray'
```

```python
def stream(Y,t,p):
    x,s = Y
    dxdt =  x*(1-x)*s/(p[0]+s) - p[1]*x
    dsdt =  p[2]*np.power(np.maximum(1-x,0),p[4])*(1-s) - p[3]*x*(1-x)*s/(p[0]+s)
    return [dxdt,dsdt]

def rates(Y,t,p):
    x,s = Y
    x,s = np.array(x),np.array(s)
    dqdt =  p[2]*np.power(np.maximum(1-x,0),p[4])*df['Value']['Y']*df['Value']['q']
    #drdt =  p[3]*(1-x)*s/(p[0]+s)*df['Value']['q']*df['Value']['Y']*df['Value']['S0']
    drdt =  p[3]*x*(1-x)*s/(p[0]+s)*df['Value']['q']*df['Value']['Y']*df['Value']['S0']
    return [dqdt,drdt]
```

```python
df = pd.DataFrame()
df["Index"]  = ['Y','q','Ks','b','Q/V','S0','n']
df.set_index("Index",inplace=True)
df["Label"]  = [r'Y',r'\hat{q}',r'K_s',r'b',r'Q/V',r'S^0',r'n']
df["Value"]  = [0.42,10,20,0.15,1000/2000,50,1]
df["Units"]  = ["mg(X)/mg(S)","mg(S)/mg(X)·d","mg(S)/L","1/d","1/d","mg(S)/L","-"]
df["Slider"] = [wd.FloatText(value=v,description=f'${k}$',step=0.1) for k,v in zip(df["Label"],df["Value"])]

nd = pd.DataFrame()
nd["Index"]  = np.arange(0,5,1)
nd.set_index("Index",inplace=True)
nd["Label"]  = [fr"\pi_{{{s}}}" for s in nd.index]
nd["Value"]  = np.arange(10,15,1)
nd["Slider"] = [wd.FloatText(value="{:.4f}".format(v),description=f'${k}$') for k,v in zip(nd["Label"],nd["Value"])]
```

```python
def update_values():
    df["Value"] = [w.value for w in df["Slider"]]
    nd["Value"] = [w.value for w in nd["Slider"]]

def calculate_ps(change):
    nd["Slider"][0].value = df["Slider"]['Ks'].value/df["Slider"]['S0'].value
    nd["Slider"][1].value = df["Slider"]['b'].value/(df["Slider"]['q'].value*df["Slider"]['Y'].value)
    nd["Slider"][2].value = df["Slider"]['Q/V'].value/(df["Slider"]['q'].value*df["Slider"]['Y'].value)
    nd["Slider"][3].value = 1.0
    nd["Slider"][4].value = df["Slider"]['n'].value

    update_values()
```

```python
for slider in df["Slider"]:
    slider.observe(calculate_ps,'value')

calculate_ps(True)
```

```python
whitespace = wd.HTML("  <b>→</b>  ")
layout = wd.Layout(justify_content='center',align_items='center')

eqTex0 = wd.HTMLMath(r'''\begin{equation}
\begin{array}{rcl}
    \dfrac{dX}{dt} &=& Y\hat{q}X\left(1-\dfrac{X}{X_{\rm max}}\right)\dfrac{S}{K_S + S} - bX \\
    \dfrac{dS}{dt} &=& \tfrac{Q}{V}\left(1-\dfrac{X}{X_{\rm max}}\right)^n (S_\infty-S) - \hat{q}X\left(1-\dfrac{X}{X_{\rm max}}\right)\dfrac{S}{K_S + S}
\end{array}
\end{equation}''',layout=layout)

eqTex1_1 = wd.HTMLMath(r'''\begin{equation*}
\begin{array}{rcl}
    s &=& S/S_\infty \\
    x &=& X/X_{\rm max} \\
    \tau &=& \hat{q}Yt \\
\end{array}
\end{equation*}''')

eqTex1_2 = wd.HTMLMath(r'''\begin{equation*}
\begin{array}{rcl}
    \pi_0 &=& K_S/S_\infty \\
    \pi_1 &=& b/\hat{q}Y \\
    \pi_2 &=& Q/V\hat{q}Y \\
    \pi_3 &=& X_{\rm max}/S_\infty Y \\
\end{array}
\end{equation*}''')

eqTex1 = wd.HBox([eqTex1_1,whitespace,eqTex1_2],layout=layout)

eqTex2 = wd.HTMLMath(r'''\begin{equation}
\begin{array}{rcl}
    \dfrac{dx}{d\tau} &=& x(1-x)\dfrac{s}{\pi_0 + s} - \pi_1 x \\
    \dfrac{ds}{d\tau} &=& \pi_2 \left(1-x\right)^n \left(1-s\right) - \pi_3 x(1-x)\dfrac{s}{\pi_0 + s}\\
\end{array}
\end{equation}''',layout=layout)

eqTex3 = wd.HTMLMath(r'''\begin{equation}
\begin{array}{cc}
    x=0 & s=1 \\
    x = \mathsf{:S} & s = \mathsf{>.<}
\end{array}
\end{equation}''',layout=layout)

eqTexs = [eqTex0,eqTex1,eqTex2,eqTex3]
```

```python
kw_streams = {'density':1,'linewidth':1.5,'arrowsize':0.8,
              'arrowstyle':'->','color':[1,1,1,0.5],'minlength':0.2}

output = wd.Output(layout=layout)

@output.capture(clear_output=True)
def plotStream():
    update_values()

    time = np.arange(0,50,0.5)
    xlin = np.linspace(0,1,61)
    slin = np.linspace(0,1,61)
    x,s  = np.meshgrid(xlin,slin)
    p = nd["Value"]
    initVals = [v.value for v in init]

    ## Calculate vector field
    dx,ds = stream([x,s],None,p)

    ## Calculate trayectories given initial condition
    tray = odeint(stream,initVals,time,args=(p,))

    ## Calculate steady-state
    #ssx,sss = fsolve(stream, initVals,args=(_,p))
    sol = root(stream,[tray[-1,0],tray[-1,1]],args=(None,p))
    ssx,sss = sol.x

    ## Calculate independent rates
    rate = rates(tray.T,None,p)

    kw_streams = {'density':2,'linewidth':1.5,'arrowsize':0.8,
              'arrowstyle':'->','color':'k','minlength':0.1}

    fig = plt.figure(figsize=[12,6])
    gs = gridspec.GridSpec(6, 12)
    ax = fig.add_subplot(gs[:,:6])
    ax.set_aspect("equal")
    ax.streamplot(x,s,dx,ds,**kw_streams,zorder=2)
    ax.scatter([0,ssx],[1,sss],s=250,clip_on=False,ec='k',label="Steady-states",zorder=3)
    ax.scatter(tray[0,0],tray[0,1],s=250,marker='X',clip_on=False,ec='k',
               label=r"$(x_0,s_0)$",zorder=4,c='wheat')

    if ssx < 0.0001:
        ax.annotate("Washout",
                  xy=(0.0, 0.99), xycoords='data',
                  xytext=(0.5, 0.5), textcoords='data',
                  size=20, va="center", ha="center",
                  bbox=dict(boxstyle="round4", fc="pink", ec="red"),
                  arrowprops=dict(arrowstyle="-|>",
                                  connectionstyle="arc3,rad=-0.2",
                                  fc="w",ec='red'))

    ax.plot(tray.T[0],tray.T[1],lw=5,c='purple')

    ax.grid(True,ls='dashed',lw=1,c=[0.5,0.5,0.5,0.5])
    ax.set_xlabel("$x$ [-]",)
    ax.set_ylabel("$s$ [-]")
    ax.set(xlim=[0,1],ylim=[0,1])
    ax.legend(loc='upper right',fontsize=10)

    pax = fig.add_subplot(gs[:3,7:])
    pax.set_facecolor("#F5DEB343")
    for variable in tray.T:
        pax.plot(time,variable,lw=3)
    pax.set_ylim([0,1])
    pax.set_xlim(left=0)
    pax.set_ylabel(r'Non-dim concentration',fontsize=8)
    pax.axes.get_xaxis().set_visible(False)
    pax.legend([r'$x(\tau)$',r'$s(\tau)$'],fontsize=10)

    rax = fig.add_subplot(gs[3:,7:])
    rax.sharex(pax)
    rax.set_facecolor("white")
    rax2 = rax.twinx()

    p1, = rax.plot(time,rate[0],lw=3,label=r'$\frac{Q}{V}(\tau) \: \mathsf{[m^3/d]}$',c='b')
    p2, = rax2.plot(time,rate[1],lw=3,label=r'$r_{\rm ut}(\tau) \: \mathsf{[mg/d/m^3]}$',c='r')
    rax.legend(handles=[p1,p2],fontsize=10)

    rax.set_xlim(left=0)
    rax.set_ylim(bottom=0)
    rax2.set_ylim(bottom=0)

    rax.set_xlabel(r'$\tau$')
    rax.set_ylabel(r'Dilution rate $Q/V$',fontsize=8)
    rax2.set_ylabel(r'Rate substrate util. $r_{\rm ut}$',fontsize=8)
    plt.show();

    return None

def updateFig(b):
    plotStream();
```

```python
x_init = wd.FloatSlider(value=0.01,min=0,max=1.0,step=0.01,description=r"$x_0$")
s_init = wd.FloatSlider(value=0.99,min=0,max=1.0,step=0.01,description=r"$s_0$")
init = [x_init,s_init]

updateFig(True);
```

```python
plotButton = wd.Button(description="Plot!",icon='drafting-compass')
plotButton.on_click(updateFig)

gs = wd.GridspecLayout(12,2)
gs[0:5,0] = wd.VBox(list(df["Slider"]),layout=wd.Layout(align_items='stretch'))
gs[0:4,1] = wd.VBox(list(nd["Slider"])+[x_init,s_init])
gs[4,1] = plotButton
gs[5:,:] = wd.HBox([output])

accordion = wd.Accordion(eqTexs,layout=wd.Layout(align_items='stretch'))
accordion.selected_index = None
accordion.set_title(0,'Governing ODEs')
accordion.set_title(1,'Nondimensionalization')
accordion.set_title(2,'Non-dimensional ODEs')
accordion.set_title(3,'Steady-state solution')

mainBox = wd.VBox([accordion,gs])
display(mainBox)
```

    VBox(children=(Accordion(children=(HTMLMath(value='\\begin{equation}\n\\begin{array}{rcl}\n    \\dfrac{dX}{dt}…


 ILI285 - Computación Científica I  / INF285 - Computación Científica

 Polynomial Interpolation: Vandermonde, Lagrange, Newton, Chebyshev

 [[S]cientific [C]omputing [T]eam](#acknowledgements)

 Version: 1.24

## Table of Contents
* [Introduction](#intro)
* [Vandermonde Matrix](#vander)
* [Lagrange Interpolation](#lagrange)
* [Runge Phenomenon](#runge)
* [Newton's Divided Difference](#DDN)
* [Interpolation Error](#Error)
* [Chebyshev Interpolation](#cheby)
* [Python Modules and Functions](#py)
* [Acknowledgements](#acknowledgements)

```python
import numpy as np
import matplotlib.pyplot as plt
import sympy as sp
from functools import reduce
import matplotlib as mpl
mpl.rcParams['font.size'] = 14
mpl.rcParams['axes.labelsize'] = 20
mpl.rcParams['xtick.labelsize'] = 14
mpl.rcParams['ytick.labelsize'] = 14
%matplotlib inline
from ipywidgets import interact, fixed
```

## Introduction

Hello! In this notebook we will learn how to interpolate 1D data with polynomials. A polynomial interpolation consists in finding a polynomial that fits a discrete set of known data points, allowing us to construct new data points within the range of the data. Formally, a polynomial $P(x)$ interpolate the data $(x_1,y_1),...,(x_n,y_n)$ if $P(x_i)=y_i$ for all $i$ in $1,...,n$.

```python
def Y(D, xi):
    # Function that evaluates the xi's points in the polynomial
    if D['M']=='Vandermonde':
        P = lambda i: i**np.arange(len(D['P']))
    elif D['M']=='Lagrange':
        P = lambda i: [np.prod(i - np.delete(D['x'],j)) for j in range(len(D['x']))]
    elif D['M']=='Newton':
        P = lambda i: np.append([1],[np.prod(i-D['x'][:j]) for j in range(1,len(D['P']))])
    return [np.dot(D['P'], P(i)) for i in xi]

def Interpolation_Plot(D):
    # Function that shows the data points and the function that interpolates them.
    xi = np.linspace(min(D['x']),max(D['x']),100)
    yi = Y(D,xi)
    plt.figure(figsize=(8,8))
    plt.plot(D['x'],D['y'],'ro',label='Interpolation points')
    plt.plot(xi,yi,'b-',label='$P(x)$')
    plt.xlim(min(xi)-0.5, max(xi)+0.5)
    plt.ylim(min(yi)-0.5, max(yi)+0.5)
    plt.grid(True)
    plt.legend(loc='best')
    plt.xlabel('$x$')
    #plt.ylabel('$P(x)$')
    plt.show()
```

## Vandermonde Matrix

First, we are going to learn the Vandermonde Matrix method. This is a $m \times m$ matrix (with $m$ being the length of the set of known data points) with the terms of a geometric progression in each row. It allows us to construct a system of linear equations with the objective of find the coefficients of the polynomial function that interpolates our data.

Example:

Given the set of known data points: $(x_1,y_1),(x_2,y_2),(x_3,y_3)$

Our system of linear equations will be:

$$ \begin{bmatrix}
    1 & x_1 & x_1^2  \\[0.3em]
    1 & x_2 & x_2^2  \\[0.3em]
    1 & x_3 & x_3^2  \end{bmatrix}
    \begin{bmatrix}
    a_1  \\[0.3em]
    a_2  \\[0.3em]
    a_3  \end{bmatrix} =
    \begin{bmatrix}
    y_1  \\[0.3em]
    y_2  \\[0.3em]
    y_3  \end{bmatrix}$$

And solving it we will find the coefficients $a_1,a_2,a_3$ that we need to construct the polynomial $P(x)=a_1+a_2x+a_3x^2$ that interpolates our data.

```python
def Vandermonde(x, y, show=False):
    # We construct the matrix and solve the system of linear equations
    A = np.array([xi**np.arange(len(x)) for xi in x])
    b = y
    xsol = np.linalg.solve(A,b)

    # The function shows the data if the flag is true
    if show:
        print('Data Points: '); print([(x[i],y[i]) for i in range(len(x))])
        print('A = '); print(np.array_str(A, precision=2, suppress_small=True))
        print("cond(A) = "+str(np.linalg.cond(A)))
        print('b = '); print(np.array_str(b, precision=2, suppress_small=True))
        print('x = '); print(np.array_str(xsol, precision=2, suppress_small=True))
        xS = sp.Symbol('x')
        F = np.dot(xS**np.arange(len(x)),xsol)
        print('Interpolation Function: ')
        print('F(x) = ')
        print(F)

    # Finally, we return a data structure with our interpolating polynomial
    D = {'M':'Vandermonde',
         'P':xsol,
         'x':x,
         'y':y}
    return D
```

```python
def show_time_V(epsilon=0):
    x = np.array([1.0,2.0,3.0+epsilon,5.0,6.5])
    y = np.array([2.0,5.0,4.0,6.0,2.0])
    D = Vandermonde(x,y,True)
    Interpolation_Plot(D)
interact(show_time_V,epsilon=(-1,2,0.1))
```

## Lagrange Interpolation

With this method, we can interpolate data thanks to the Lagrange basis polynomials. Given a set of $n$ data points $(x_1,y_1),...,(x_n,y_n)$, the Lagrange interpolation polynomial is the following:

$$ P(x) = \sum^n_{i=1} y_i\,L_i(x),$$

where $L_i(x)$ are the Lagrange basis polynomials:

$$ L_i(x) = \prod^n_{j=1,j \neq i} \frac{x-x_j}{x_i-x_j} = \frac{x-x_1}{x_i-x_1} \cdot ... \cdot \frac{x-x_{i-1}}{x_i-x_{i-1}} \cdot \frac{x-x_{i+1}}{x_i-x_{i+1}} \cdot ... \cdot \frac{x-x_n}{x_i-x_n}$$

or simply $L_i(x)=\dfrac{l_i(x)}{l_i(x_i)}$, where $l_i(x)=\displaystyle{\prod^n_{j=1,j \neq i} (x-x_j)}$.

The most important property of these basis polynomials is:

$$ L_{j \neq i}(x_i) = 0 $$

$$ L_i(x_i) = 1 $$

So, we assure that $L(x_i) = y_i$, which indeed interpolates the data.

```python
def Lagrange(x, y, show=False):
    # We calculate the li's
    p = np.array([y[i]/np.prod(x[i] - np.delete(x,i)) for i in range(len(x))])

    # The function shows the data if the flag is true
    if show:
        print('Data Points: '); print([(x[i],y[i]) for i in range(len(x))])
        xS = sp.Symbol('x')
        L = np.dot(np.array([np.prod(xS - np.delete(x,i))/np.prod(x[i] - np.delete(x,i)) for i in range(len(x))]),y)
        print('Interpolation Function: ');
        print(L)

    # Finally, we return a data structure with our interpolating polynomial
    D = {'M':'Lagrange',
         'P':p,
         'x':x,
         'y':y}
    return D
```

```python
def show_time_L(epsilon=0):
    x = np.array([1.0,2.0,3.0+epsilon,4.0,5.0,6.0,7.0])
    y = np.array([2.0,5.0,4.0,6.0,7.0,8.0,3.0])
    D = Lagrange(x,y,True)
    Interpolation_Plot(D)
interact(show_time_L,epsilon=(-1,1,0.1))
```

Here you get some questions about Lagrange Interpolation:
- Explain what happens with the interpolator polynomial when you add a new point to the set of points to interpolate. **Answer: We need to recalculate the polynomial**
- Why it is not a good idea to use Lagrange interpolation for a set of points which is constantly changing? **A: Because we need to compute the whole interpolation again**
- What is the operation count of obtaining the interpolator polynomial using Lagrange? What happens with the error?

## Newton's Divided Difference

In this interpolation method we will use divided differences to calculate the coefficients of our interpolation polynomial. Given a set of $n$ data points $(x_1,y_1),...,(x_n,y_n)$, the Newton polynomial is:

$$ P(x) = \sum^n_{i=1} (f[x_1 ... x_i] \cdot \prod^{i-1}_{j=1} (x-x_j)) ,$$

where $ \prod^{0}_{j=1} (x-x_j) = 0 $, and:

$$ f[x_i] = y_i $$

$$ f[x_j...x_i] = \frac{f[x_{j+1}...x_i]-f[x_j...x_{i-1}]}{x_i-x_j}$$

```python
def Divided_Differences(x, y):
    dd = np.array([y])
    for i in range(len(x)-1):
        ddi = []
        for a in range(len(x)-i-1):
            ddi.append((dd[i][a+1]-dd[i][a])/(x[a+i+1]-x[a]))
        ddi = np.append(ddi,np.full((len(x)-len(ddi),),0.0))
        dd = np.append(dd,[ddi],axis=0)
    return np.array(dd)

def Newton(x, y, show=False):
    # We calculate the divided differences and store them in a data structure
    dd = Divided_Differences(x,y)

    # The function shows the data if the flag is true
    if show:
        print('Data Points: '); print([(x[i],y[i]) for i in range(len(x))])
        xS = sp.Symbol('x')
        N = np.dot(dd[:,0],np.append([1],[np.prod(xS-x[:i]) for i in range(1,len(dd))]))
        print('Interpolation Function: ');
        print(N)

    # Finally, we return a data structure with our interpolating polynomial
    D = {'M':'Newton',
         'P':dd[:,0],
         'x':x,
         'y':y}
    return D
```

```python
def show_time_N(epsilon=0):
    x = np.array([0.0,2.0,3.0+epsilon,4.0,5.0,6.0])
    y = np.array([1.0,3.0,0.0,6.0,8.0,4.0])
    D = Newton(x,y,True)
    Interpolation_Plot(D)
interact(show_time_N,epsilon=(-1,1,0.1))
```

Questions about Newton's DD:
- What is the main problem using this method (and Lagrange)? How can you fix it? **A: A problem with polynomial interpolation with equispaced date is the Runge phenomenon and can be handle with Chebyshev points**
- What to do when a new point is added? **A: Pro, is not necessary re-calculate the whole polynomial only a small piece**

## Polynomial Interpolation Error

The interpolation error is given by:

$$ f(x)-P(x) = \frac{(x-x_1) \cdot (x-x_2) \cdot ... \cdot (x-x_n)}{n!} \cdot f^{(n)}(c) ,$$

where $c$ is within the interval from the minimun value of $x$ and the maximum one.

```python
def Error(f, n, xmin, xmax, method=Lagrange, points=np.linspace, plot_flag=True):
    # This function plots f(x), the interpolating polynomial, and the associated error
    # points can be np.linspace to equidistant points or Chebyshev to get Chebyshev points
    x = points(xmin,xmax,n)
    y = f(x)

    xe = np.linspace(xmin,xmax,100)
    ye = f(xe)

    D = method(x,y)
    yi = Y(D, xe)

    if plot_flag:
        plt.figure(figsize=(5,10))
        f, (ax1, ax2) = plt.subplots(1, 2, figsize=(15,5), sharey = False)

        ax1.plot(xe, ye,'r-', label='f(x)')
        ax1.plot(x, y,'ro', label='Interpolation points')
        ax1.plot(xe, yi,'b-', label='Interpolation')
        ax1.set_xlim(xmin-0.5,xmax+0.5)
        ax1.set_ylim(min(yi)-0.5,max(yi)+0.5)
        ax1.set_title('Interpolation')
        ax1.grid(True)
        ax1.set_xlabel('$x$')
        ax1.legend(loc='best')

        ax2.semilogy(xe, abs(ye-yi),'b-', label='Absolute Error')
        ax2.set_xlim(xmin-0.5,xmax+0.5)
        ax2.set_title('Absolute Error')
        ax2.set_xlabel('$x$')
        ax2.grid(True)
        #ax2.legend(loc='best')
        plt.show()

    return max(abs(ye-yi))
```

```python
def test_error_Newton(n=5):
    #me = Error(lambda x: np.sin(x)**3, n, 1, 7, Newton)
    me = Error(lambda x: (1/(1+12*x**2)), n, -1, 1, Newton)
    print("Max Error:", me)
```

```python
interact(test_error_Newton,n=(5,25))
```

##  **Runge's Phenomenon**: It is a problem of oscillation of polynomials at the edges of the interval.

We are interpolating a data that is 0 almost everywhere and 1 at the middle point, notice that when $n$ increases the oscilations increase and the red dots seems to be at 0 everywhere but it is just an artifact, there must be a 1 at the middle. The oscillations you see at the end of the interval is the Runge phenomenon.

```python
def Runge(n=9):
    x = np.linspace(0,1,n)
    y = np.zeros(n)
    y[int((n-1.0)/2.)]=1
    D = Newton(x,y,False)
    Interpolation_Plot(D)
interact(Runge,n=(5,25,2))
```

## Chebyshev Interpolation

With the objective of reducing the error of the polynomial interpolation, we need to find the values of $x_1,x_2,...,x_n$ that minimize $(x-x_1) \cdot (x-x_2) \cdot ... \cdot (x-x_n)$.

To choose these values of $-1 \leq x_1,x_2,...,x_n \leq 1$ (to use another interval we just need to do a change of variables) that minimize the error, we will use the roots of the Chebyshev polynomials, also called **Chebyshev nodes**, which are defined by:

$$ x_i = \cos\left(\frac{(2i-1)\pi}{2n}\right), i = 1,...,n $$

```python
def Chebyshev(xmin,xmax,n=5):
    # This function calculates the n Chebyshev points and plots or returns them depending on ax
    ns = np.arange(1,n+1)
    x = np.cos((2*ns-1)*np.pi/(2*n))
    y = np.sin((2*ns-1)*np.pi/(2*n))
    plt.figure(figsize=(10,5))
    plt.ylim(-0.1,1.1)
    plt.xlim(-1.1,1.1)
    plt.plot(np.cos(np.linspace(0,np.pi)),np.sin(np.linspace(0,np.pi)),'k-')
    plt.plot([-2,2],[0,0],'k-')
    plt.plot([0,0],[-1,2],'k-')
    for i in range(len(y)):
        plt.plot([x[i],x[i]],[0,y[i]],'r-')
        plt.plot([0,x[i]],[0,y[i]],'r-')
    plt.plot(x,[0]*len(x),'bo',label='Chebyshev points')
    plt.plot(x,y,'ro')
    plt.xlabel('$x$')
    plt.title('n = '+str(n))
    plt.grid(True)
    plt.legend(loc='best')
    plt.show()
def Chebyshev_points(xmin,xmax,n):
    ns = np.arange(1,n+1)
    x = np.cos((2*ns-1)*np.pi/(2*n))
    y = np.sin((2*ns-1)*np.pi/(2*n))
    return (xmin+xmax)/2 + (xmax-xmin)*x/2
```

```python
interact(Chebyshev,xmin=fixed(-1),xmax=fixed(1),n=(2,50))
```

By using these points, we reduce the numerator of the interpolation error formula:

$$ (x-x_1) \cdot (x-x_2) \cdot ... \cdot (x-x_n) = \dfrac{1}{2^{n-1}} \cdot T_n(x), $$

where $T(x) = \cos (n \cdot \arccos (x))$ is the n-th Chebyshev polynomial.

$$ T_0(x) = 1 $$
$$ T_1(x) = x $$
$$ T_2(x) = 2x^2 -1 $$
$$...$$
$$ T_{n+1}(x) = 2 \cdot x \cdot T_n(x) - T_{n-1}(x) $$

```python
def T(n,x):
    # Recursive function that returns the n-th Chebyshev polynomial evaluated at x
    if n == 0:
        return x**0
    elif n == 1:
        return x
    else:
        return 2*x*T(n-1,x)-T(n-2,x)

def Chebyshev_Polynomials(n=2, Flag_All_Tn=False):
    # This function plots the first n Chebyshev polynomials
    x = np.linspace(-1,1,1000)
    plt.figure(figsize=(10,5))
    plt.xlim(-1, 1)
    plt.ylim(-1.1, 1.1)
    if Flag_All_Tn:
        for i in np.arange(n+1):
            y = T(i,x)
            plt.plot(x,y,label='$T_{'+str(i)+'}(x)$')
    else:
        y = T(n,x)
        plt.plot(x,y,label='$T_{'+str(n)+'}(x)$')
    #    plt.title('$T_${:}$(x)$'.format(n))
    plt.legend(loc='right')
    plt.grid(True)
    plt.xlabel('$x$')
    plt.show()
```

```python
#Chebyshev_Polynomials(3,True)
interact(Chebyshev_Polynomials,n=(0,12),Flag_All_Tn=True)
```

```python
n=9
xmin=1
xmax=9

mee = Error(lambda x: np.sin(x)**3, n, xmin, xmax, method=Lagrange)
mec = Error(lambda x: np.sin(x)**3, n, xmin, xmax, method=Lagrange, points=Chebyshev_points)

print("Max error (equidistants points):", mee)
print("Max error (Chebyshev nodes):", mec)
```

```python
def test_error_chebyshev(n):
    mee = Error(lambda x: (1/(1+12*x**2)), n, -1, 1, Lagrange)
    mec = Error(lambda x: (1/(1+12*x**2)), n, -1, 1, method=Lagrange, points=Chebyshev_points)
    print("Max error (equidistants points):", mee)
    print("Max error (Chebyshev nodes):", mec)
```

```python
interact(test_error_chebyshev,n=(5,25))
```

Questions about Chebyshev:
- How can you calculate the Chebyshev points in the interval [a,b] instead of [-1,1]? **A: Using a change of variable**

## Convergence analysis

```python
n=50
shift=2

my_functions={0:lambda x: (x)**10,
              1:lambda x: np.abs((x)**3),
              2:lambda x: np.exp(-((x)**-2)),
              3:lambda x: 1/(1+x**2),
              4:lambda x: np.sin(x)**3}
labels = {0: "x^{10}",
          1: "|x^3|",
          2: "\exp(-x^2)",
          3: "1/(1+x^2)",
          4: "\sin^3(x)"}

n_points=np.arange(shift,n)

for k in np.arange(5):
    max_error=np.zeros(n-shift)
    max_error_es=np.zeros(n-shift)
    for i in n_points:
        max_error[i-shift] = Error(my_functions[k], i, -1, 1, Newton, Chebyshev_points, plot_flag=False)
        max_error_es[i-shift] = Error(my_functions[k], i, -1, 1, Newton, points=np.linspace, plot_flag=False)
    axis=plt.figure()
    plt.semilogy(n_points,max_error,'kd',label='Chebyshev points')
    plt.semilogy(n_points,max_error_es,'k.',label='Equalspaced poins')
    plt.ylim(10**-16,10**4)
    plt.grid(True)
    plt.title('Interpolation Error of $f(x)='+str(labels[k])+"$")
    plt.xlabel('Number of points used in the interpolation')
    plt.ylabel('Max error on domain')
    plt.legend(loc='best')
    plt.show()
```

## Python Modules and Functions

Interpolation:

http://docs.scipy.org/doc/numpy-1.10.0/reference/generated/numpy.polyfit.html

Vandermonde Matrix:

http://docs.scipy.org/doc/numpy-1.10.0/reference/generated/numpy.vander.html

Lagrange:

http://docs.scipy.org/doc/scipy-0.14.0/reference/generated/scipy.interpolate.lagrange.html

Chebyshev Points:

http://docs.scipy.org/doc/numpy-1.10.1/reference/generated/numpy.polynomial.chebyshev.chebroots.html#numpy.polynomial.chebyshev.chebroots

# Acknowledgements
* _Material created by professor Claudio Torres_ (`ctorres@inf.utfsm.cl`) _and assistants: Laura Bermeo, Alvaro Salinas, Axel Simonsen and Martín Villanueva. DI UTFSM. April 2016._
* _Material modified by Cristopher Arenas. May 2017._
* _Material modified by Claudio Torres. May 2017._
* _Bug fixed by Cristobal Carmona. April 2018._

```python

```


Make sure you remove `raise NotImplementedError()` and fill in any place that says `# YOUR CODE HERE`, as well as your `NAME`, `ID`, and `LAB_SECTION` below:

```python
NAME = "FAHMID BIN KIBRIA"
ID = "19201063"
LAB_SECTION = "02"
```

---

Polynomial Root Finding Using Bisection Method
---

### `pandas` Dataframe:
Before we start we will know a little about `pandas` dataframe. `pandas` is a python library. Dataframes are classes that are used to store complex data. You can initialize one as a python dictionary. Let's take a look.

```python
import pandas as pd

x = [0, 1, 2, 3, 4, 5, 6, 7]
y = [1.0, 5.2, 3.9, 8.4, 14.6, 12.3, 8.9, 10.2]

dictionary = {
    "x": x,
    "y": y,
}
df = pd.DataFrame(dictionary)
df.head(8)
```

      x
      y

      0
      0
      1.0

      1
      1
      5.2

      2
      2
      3.9

      3
      3
      8.4

      4
      4
      14.6

      5
      5
      12.3

      6
      6
      8.9

      7
      7
      10.2

We can use it to print data in a tabular format. We can even use more numpy arrays with it.

```python
import numpy as np

x = np.linspace(0, np.pi/2, 15)
y = np.sin(x)

dictionary = {
    "x": x,
    "sin(x)": y,
}
df = pd.DataFrame(dictionary)
df.head(15)
```

      x
      sin(x)

      0
      0.000000
      0.000000

      1
      0.112200
      0.111964

      2
      0.224399
      0.222521

      3
      0.336599
      0.330279

      4
      0.448799
      0.433884

      5
      0.560999
      0.532032

      6
      0.673198
      0.623490

      7
      0.785398
      0.707107

      8
      0.897598
      0.781831

      9
      1.009798
      0.846724

      10
      1.121997
      0.900969

      11
      1.234197
      0.943883

      12
      1.346397
      0.974928

      13
      1.458597
      0.993712

      14
      1.570796
      1.000000

### Root Finding:
Let $f(x)$ be a function of $x$. if for some $x=\alpha, f(x) = f(\alpha) = 0$, we say $\alpha$ is a root of function $x$.

Let,
$$f(x) = x^5 + 2.5x^4 - 2x^3 -6x^2 + x + 2\tag{6.1}$$

The graph of $f(x)$ looks like this.

```python
from numpy.polynomial import Polynomial
import matplotlib.pyplot as plt

f = Polynomial([2.0, 1.0, -6.0, -2.0, 2.5, 1.0])
x = np.linspace(-2.5, 1.5, 100)
y = f(x)
dictionary = {
    'x': x,
    'y': y
}
plt.axhline(y=0, color='k')
plt.plot(x, y)
plt.plot(f.roots(), [0.0]*5, 'ro')
print(f.roots())
```

Roots are the points where the graph intersects with the $X$-axis.

### Bisection

One way to find out root's are to use bisection method. Here is the strategy, if $\alpha$ is a root between and interval $[a, b]$ then graph will cross the $X$-axis at $\alpha$. So, $sign( f(\alpha-h) ) = - sign( f(\alpha+h) )$, for small value of $h$.

So, we can work our way up towards the root by taking average of $a$ and $b$, as long as the signs are different.

we will start with $a_0$ and $b_0$, such that, $f(a_0) f(b_0) < 0$.
Then we iterate as this,
\begin{align}
m_k &= \frac{a_k + b_k}{2} \\
\text{if, } f(a_k) f(m_k) < 0, \text{ then, } a_{k+1} &= a_k \text{ and } b_{k+1} = m_k\\
\text{else, } a_{k+1} &= m_k \text{ and, } b_{k+1} = b_k
\end{align}

We keep iterating until we find the root with sufficient precision. We usually use a formula like this,
$$
\frac{|m_{k+1} - m_k|}{|m_{k+1}|} \leq \epsilon \\
$$
Where, $\epsilon$ is a very small value, like $\epsilon < 10^{-6}$

### Complete the code below
Complete the code below to iterate and solve for a root of the following equation, between the interval, $[-0.5, 1.3]$:
\begin{aligned}
    f(x) = 2 + x - 6x^2 - 2x^3 + 2.5x^4 + x^5.
\end{aligned}

```python
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from numpy.polynomial import Polynomial

# The polynomial and the range is defined for you
f = Polynomial([2.0, 1.0, -6.0, -2.0, 2.5, 1.0])
a = -0.5
b = 1.3
m = (a + b) / 2
e = 1e-6

root = 0.0    # You need to update this value

# Populate the following lists in each iteration
list_a = []
list_b = []
list_m = []
list_f = []

# YOUR CODE HERE
def m_k(a_k, b_k):
    return ( a_k + b_k ) / 2

list_a.append(a)
list_b.append(b)
list_m.append(m)
list_f.append( f(a) * f(m) )

k = 0
loop = True
while loop:
    if list_f[k]<0:
        list_a.append(list_a[k])
        list_b.append(list_m[k])
        list_m.append( m_k(list_a[k+1], list_b[k+1]) )
        list_f.append( f(list_a[k+1]) * f(list_m[k+1]) )

    else:
        list_a.append(list_m[k])
        list_b.append(list_b[k])
        list_m.append( m_k(list_a[k+1], list_b[k+1]) )
        list_f.append( f(list_a[k+1]) * f(list_m[k+1]) )

    if abs( list_m[k+1] - list_m[k] ) / abs( list_m[k+1] ) <= e:
        root = list_m[k]
        break

    k+=1
```

```python
xs = np.linspace(-2.5, 1.5, 100)
ys = f(xs)

plt.axhline(y=0, color='k')
plt.plot(xs, ys)
plt.plot(root, f(root), 'ro')

print(pd.DataFrame({'a':list_a, 'b':list_b, 'm':list_m, 'f(m)':list_f}))

assert "{:.3f}".format(root) == "0.672"
```

```python

```